Press Space to continue
Production judges find failures offline datasets miss
Press Space to continue
the useful part of a judge on production traffic is what happens to the outputs it flags calibrate it on human labels, flag low scorers, confirm the real failures, add them to evals, fix, rerun prod finds the failures your offline dataset never had
@PyTorchQuoted post unavailable hereOpen on Xopen on x · 1 viewViews and clicks count once per visitor per day. Clicks are opens of this post on X.