Build aligned Judges and Rankers
Start building with prompts. Align with less than 30 labels.
Leverage the power of rubrics
Use stable success criteria instead of unstable prompts to tune decision-making.
Pi Score:
0.90
0.70
Avoids Legal Jargon
Does the summary avoid unnecessary legal jargon while retaining essential legal terminology?
0.90
Clarity
Is the summary written in clear and understandable language suitable for a general audience?
Flexible natural language criteria is evaluated with Pi Scorer, our state of the art scoring model designed for judging data.
Upload sources
Extract insights from usage data to determine what signals you're missing.
Align Rubrics
Determine what criteria needs to be evaluated to make choices like your human raters.
Label high-leverage data
Find the most important data to label and generate new rubrics with improved alignment.
Judge and rank just like your users and experts.
Upload unrated logs, requirements, or preferences.
Generate an aligned rubric using our genetic algorithm.
Find faulty criteria by testing your judge against unlabeled and rated data.
Come back once you've collected more data, and customize your judge even further.
Mine labels, logs, PRDs, and system prompts for requirements that actually matter.
Build, prune, and shape the corpus of data that most effectively aligns your judge.
Measure alignment and criteria health against real data to find ways to improve your rubric.
Find the most important data to label. Filter thousands of logs instantly to five you need to review.
Pi's foundation models ensure judgements are consistent even when criteria is edited.
