Evaluation that evolves with your AI
Create a tunable scoring system that integrates natural language and code-based criteria into your AI application
Trusted By
Why score with Pi?
We turn your evaluations into precise, user-calibrated, and cost-effective signals for use anywhere in your stack.
Transform data into metrics
Not sure what to measure? Pi figures it out for you. Feed it any or all of your prompts, your PRDs, your user feedback, or just sit down and chat with it and it will help you figure out the best calibrated metrics for your application.
Quick, deterministic scores
Tap to view
Framework agnostic
Tap to view
A foundation model designed for scoring
We train our models to understand principles, not mimic content. We continuously monitor performance to improve quality with each release.
Aligned with your users & experts.
The best metrics align with human judgment. You can continuously improve your Pi
scoring system by calibrating it on your own labels, preferences, and user data,
adjusting to match your team's expertise and actual user behavior in a virtuous
feedback loop.
Fully captures correctness and taste.
Pi’s scoring system combines soft measures like natural language quality, hard
measures like code correctness, and trained measures like thumbs-up prediction. This
comprehensiveness gives you the highest quality evals, reward models, ranking
functions, and agent decision nodes.
5x cheaper than LLM judges.
Maintaining the performance of a large model on a smaller size means you can afford to
measure all that matters to you without running a massive bill. You can reinvest your
savings to measure even more dimensions, more frequently, across your workflows.
from withpi import PiClient
pi = PiClient()
scores = pi.scoring_system.score(
llm_input="Pi Labs",
llm_output="Score anything with Pi Labs today!",
scoring_spec=[{"question": "Is there a strong call to action?"}]
)
print(scores.total_score)
