/ Studio
Build aligned Judges and Rankers
Start building with prompts. Align with less than 30 labels.
Leverage the power of rubrics
Use stable success criteria instead of unstable prompts to tune decision-making.
Total Score:
0.90
0.70
Avoids Legal Jargon
Does the summary avoid unnecessary legal jargon while retaining essential legal terminology?
0.90
Clarity
Is the summary written in clear and understandable language suitable for a general audience?
Flexible natural language criteria is evaluated with Pi Scorer, our state of the art scoring model designed for judging text data.
Upload sources
Extract insights from usage data to determine what signals you're missing.
Align Rubrics
Determine what criteria needs to be evaluated to make choices like your human raters.
Label high-leverage data
Find the most important data to label and generate new rubrics with improved alignment.
Understand any data
Mine labels, logs, PRDs, and system prompts for requirements that actually matter.
Experiment with different sources
Build, prune, and shape the corpus of data that most effectively aligns your judge.
Diagnose judgment issues
Measure alignment and criteria health against real data to find ways to improve your rubric.
Label high-leverage data
Find the most important data to label. Filter thousands of logs instantly to five you need to review.
Make consistent improvements
Pi's foundation models ensure judgements are consistent even when criteria is edited.
Understand automated judgements
Explain any scored example by reviewing your rubric's individual criteria scores.
Judge and rank just like your users.
Upload unrated logs, requirements, or preferences.
Generate an aligned rubric automatically, using our genetic algorithm.
Find faulty criteria and tricky examples by testing your judge against unlabeled and rated data.
Come back once you've collected more data, and customize your judge even further.
