Llm Evaluator (Model Response Analyst)
Location
madrid, kingdom of spain
Job Type
Full-time
Category
Social Care
Posted
June 02, 2026
Job Title:
¿Tiene lo que se necesita para triunfar? La siguiente información debe ser leída atentamente por todos los candidatos.
LLM Evaluator (Model Response Analyst)
Location:
Remote (Worldwide)
Job Summary:
We are seeking a detail-oriented and analytical LLM Evaluator to assess, analyze, and improve the performance of large language models (LLMs). In this role, you will evaluate AI-generated content for accuracy, coherence, factual reliability, bias, safety, and alignment with defined guidelines.
Responsibilities
Evaluate and rank model-generated text based on complex rubrics covering dimensions such as factuality, coherence, safety, instruction‑following, and creativity.
Review multiple model responses to the same prompt and determine which output a human would prefer, providing justifications for your choices.
Provide clear, concise feedback to the modeling and training teams regarding recurring failure models observed during eval...
¿Tiene lo que se necesita para triunfar? La siguiente información debe ser leída atentamente por todos los candidatos.
LLM Evaluator (Model Response Analyst)
Location:
Remote (Worldwide)
Job Summary:
We are seeking a detail-oriented and analytical LLM Evaluator to assess, analyze, and improve the performance of large language models (LLMs). In this role, you will evaluate AI-generated content for accuracy, coherence, factual reliability, bias, safety, and alignment with defined guidelines.
Responsibilities
Evaluate and rank model-generated text based on complex rubrics covering dimensions such as factuality, coherence, safety, instruction‑following, and creativity.
Review multiple model responses to the same prompt and determine which output a human would prefer, providing justifications for your choices.
Provide clear, concise feedback to the modeling and training teams regarding recurring failure models observed during eval...