Perseval Task Completion — development runtime candidate
This repository contains the compact ONNX artifact used to exercise Perseval's local Task Completion evaluator and model-management flow.
Status
This is a development candidate, not a release-certified evaluator. It is published so the macOS application can install, verify, update, and execute a real local model end to end. Do not interpret its scores as production-quality ground truth.
The frozen private comparison holdout remains sealed. A later release will be marked certified only after it passes the documented holdout gates.
Files
manifest.json— versioned model, projector, feature, calibration, and file integrity contract.model.onnx— compact binary Task Completion head.tokenizer.json— pinned tokenizer reference.parity.json— Python/ONNX/Rust parity fixtures.catalog.json— mutable release pointer whose artifact revision is immutable.
Perseval downloads files from the immutable revision named in catalog.json,
checks every size and SHA-256 digest, then validates the internal manifest
before making the installation active.
Runtime contract
- Projector:
traceeval.compact-task-completion-projector.v1 - Structured feature schema:
traceeval.task_completion_structured_evidence.v2 - Inputs: 39 structured evidence features
- Outputs: binary completion logits
- Calibration threshold:
0.745
The application keeps trace projection, privacy enforcement, revision checks, hash verification, inference, evidence resolution, and threshold application in Rust.
Intended use
Use this candidate to:
- test Perseval's local learned-assessment runtime;
- validate model installation and updates without manual folder entry;
- run development benchmarks and Python/ONNX/Rust parity checks.
Do not use it as an automated decision-maker or as a substitute for human review in high-impact settings.
Training data and limitations
The development pipeline combines source-balanced agent traces, structured
proof-bearing evidence, and group-safe splits. Synthetic, redistributable
examples are published separately at
Etolith/perseval-agent-evaluator-synthetic.
Known limitations include domain shift, unsupported-success claims, sparse external-side-effect evidence, and incomplete calibration across long, multi-agent traces. The compact head consumes a projected representation, not raw screenshots or arbitrary full-context traces.
Reproducibility
The manifest pins the model, tokenizer, parity fixtures, projector, feature schema, training run, and calibration versions. The public application verifies those contracts before loading the artifact. Private traces, credentials, teacher prompts, and unreleased checkpoints are not included.