Skip to main content

Criterion judge

metis.library.graders.CriterionJudgeGrader
Measure
none
Scopes
session
Grader type
llm
Reference type
rubric_based
Judgment form
binary
Aggregation
none_single_score

Assess the recorded session against a criterion description with a binary score.

class CriterionJudgeArgs(JudgeArgs):
criterion: NonEmptyText


class CriterionJudgeGrader(Grader[CriterionJudgeArgs]):
def criteria(self) -> str:
return (
"Assess the entire recorded session against the criterion below."
f"\n\n<criterion>\n{self.args.criterion}\n</criterion>"
)