doc-img-classification · commercial

Classify any document or figure image against labels you choose at runtime. doc-img-classification is a document-adapted dual-encoder (image ↔ text) that scores an image against an arbitrary set of text labels and returns the best match — invoices, resumes, scientific papers, financial reports, receipts, charts, signatures, screenshots, and whatever label you type next. No fixed class list, no retraining to add a category: it is open-vocabulary.

It is the document specialist in a field of generalists. General open-vocab models (CLIP, SigLIP) were trained on web photos; this one is adapted to documents and figures — and it shows on the leaderboard. Weights are commercial (not downloadable here); this page is a spec + scorecard.

• 🎯 Try it: doc-openvocab-demo
• 🏆 Leaderboard: doc-openvocab-leaderboard
• 📊 Benchmark: doc-openvocab-benchmark

Results

On the open doc-openvocab-benchmark — document & figure images scored against a ~48-label open vocabulary — against the strongest general open-vocab image classifiers. Top-1 accuracy; higher is better.

Track	doc-img-classification	OpenAI CLIP L/14	LAION CLIP-H/14	SigLIP2 SO400M
Broad (48-label open vocab)	.880	.726	.654	.801
Specialized (document categories)	.712	.463	.250	.536
Synonym (paraphrase robustness)	.730	.738	.672	.689

The point of the benchmark: this is the open-vocabulary task — labels supplied at inference, not baked into a head — which a fixed-label classifier structurally cannot do. Among models that can do it, the document-adapted one leads on document and figure imagery. Full ranking on the leaderboard.

Concepts, not strings. On the synonym track the same images are scored against held-out paraphrases of every label ("invoice" → "bill", "manuals" → "user guide"). Accuracy holds — the model matches the concept, so your exact wording doesn't have to match ours.

Calibrating the score

Contrastive training maximizes ranking, which can leave the raw similarity overconfident. For a score you can threshold, apply temperature scaling — divide the logits by a fitted T before the sigmoid. It is monotonic, so ranking/accuracy is untouched and only the confidence values are repaired. On our serving distribution the fitted temperature drives pairwise calibration error from 0.027 → 0.002, and it ships folded directly into the exported graph. Re-fit T if your input pipeline changes.

Intended use & limits

• Use it for: zero-shot classification of document page images and figure/zone crops in a free-label setting — the caller supplies the candidate labels. Built for the Nutrient Maestro SDK image-classification contract (a single combined ONNX graph, on-prem).
• Limits: optimized for document and figure imagery; general natural-image zero-shot ability is preserved but not the focus. English label strings are the primary target.

License & data

The model weights are offered under a commercial Nutrient license — deployed on-prem, so your documents never leave your infrastructure. The training set is not redistributed. The evaluation data is public and reproducible: doc-openvocab-benchmark (CC-BY-SA-4.0, test-only).

📩 Get access

doc-img-classification is commercial and its weights are not downloadable here. To run it on-prem — open-vocabulary, calibrated, private — contact Nutrient: nutrient.io/contact-sales.

About the author

This project is maintained and funded by Nutrient - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks.