Pi Fundamentals | 2. Architecture

Architecture and Design

Signals and Scoring: An architecture inspired by Ranking

Break down a complex Scoring problem into simpler constituent signals.

Core Insight

Decouple two fundamental tasks in Scoring

Generate Signals

Turn Subjective Metric → Tree of Signals

Start with an LLM for language understanding, attention, and world knowledge.
Swap the Generative head with a Regression head to train for a Scoring task instead of Generation.
Remove Masking to allow the model to look at all tokens at the same time, increasing accuracy.
Train for Question Answering, not for Instruction Following.
Run inference in parallel on the whole tree (all signals) together to lower latency.
Target small models (<1B, 1B, 8B) with large context windows, as Scoring is a simpler task and the questions are designed to be simple without requiring reasoning.

Compute Signals

Calibrate signal combination against Ground Truth

Move implicit model reasoning to explicit combinations. More control, explainability, and debuggability.
Pi scoring models handle subjective dimensions. A node can also be expressed in code for objective dimensions as well as other other trained models.
GAMs provide explainable non-linear combinations (vs. other simple weighted average, or inscrutable black boxes etc.)
GAMs & ASTs can be initialized by prompting, calibrated manually by humans and eventually trained with Rater data.
Abstract Syntax Trees handle control flow for more complex decision making in scoring.

Next up

Scoring Api

Learn how Pi's scoring framework works