You can now build aligned judges and rankers with Pi Studio, a tool designed specifically for tuning rubrics with highly leveraged data. Upload requirements, unlabeled data, ratings, and preferences to generate new rubrics that are aligned against your data. Edit your rubric with confidence by running alignment and criteria health tests against it. Search thousands of logs to find high leverage data to label, and further align your rubric.

This update deprecates Pi Copilot in favor of Pi Studio. All previous Pi Copilot (deprecated) projects have been migrated to Pi Studio projects. Major changes (compared to Pi Copilot) include:

1. Scoring Spec renamed to Rubric
2. Either a Judge or Ranker can be built (instead of generic scorer)
3. Structured/unstructured sources can be uploaded and mapped in the Sources Panel
4. Rubric is generated from selected sources using an improved genetic algorithm
5. Copilot can find data to label data, read sources, and read reports.
6. Copilot can no longer edit rubrics or generate examples to score (reports, and sources achieve this).
7. Rubric alignment and criteria health can be measured against uploaded sources, and acted upon.

In this update, users can build Judges using Pi Studio. Rankers will come in the next coming updates.

Starting May 22nd, 2025, Pi Copilot is now available for free on our website. This Chatbot is specially tuned for building and integrating scoring specs from domain information specified in the chat. The chatbot quickly helps users setup a sample scoring spec, and shows them how to integrate with our SDK.

Integrations: