Veridion open-set calibration: scale-aware thresholds and measured abstention

Question — Can one cosine threshold retain the same meaning as a character gallery grows or a query changes from one whole frame to several cropped regions?

Observation — The 1:N false-positive rate of a fixed threshold rises with the number of searchable entries. Cropped queries produce a different impostor distribution from whole frames, while embedding two characters together can create a blended point belonging to neither identity.

Decision — Veridion derives its cut-point from gallery size and an FPIR budget. When the empirical impostor tail cannot resolve the required rate, it does not extrapolate: it withholds the conclusion.

Character-IP review combines questions that should remain separate. “Is this the same file after compression, cropping or transformation?” is not the same as “Does a new pose or redraw depict the same character?” The first is copy detection; the second is visual-identity retrieval. Add Korean text, narrative, speech style and persona, and any design that compresses everything into one infringement score obscures both the type and the limits of its evidence.

My work on Veridion was not the invention of a new embedding model. I solely owned 100% of the requirements analysis, system architecture and full-stack implementation, composing existing models into a reviewable evidence system. The central design separates retrieval from legal conclusions and joins scale, query shape, measurement resolution and human-review budget into one decision contract.

1. Separate what “the same” means before calibrating it

The visual path preserves two model families as different evidence channels.

Path	Question	Characteristic failure
Near-duplicate retrieval	Is this the same file or a transformation of it?	Weak to redraws and pose changes
Character-identity retrieval	Is this the same character in another scene?	Confuses castmates sharing house style and palette

Model selection began with a 240-image leave-one-out study. Fourteen characters contributed eleven images each, while 86 single-image characters served as distractors. The repository records the following results over 28,680 pairs.

Model	ROC-AUC	best F1	top-1
CCIP v2 caformer_s36	0.9852	0.723	94.2%
SigLIP2-base	0.9488	0.522	89.6%
SSCD	0.6906	0.320	55.2%
DINOv2-base	0.6873	0.226	52.6%

These are historical clean-benchmark measurements recorded in the repository, not results rerun while preparing this record. The retained source snapshot does not contain the input images, embedding cache or raw outputs. The table therefore supports a model choice for this anime/manga sample; it does not promise 94.2% on arbitrary webtoons or live traffic.

L2-normalised features and exact inner-product search were sufficient to rank candidates. Turning a high cosine into a user-facing decision was a separate problem.

2. A fixed cosine changes meaning as the gallery grows

Let FMR be the probability that one nonmatching comparison crosses threshold τ. When one query searches N entries, the probability that at least one nonmatch crosses the line is the FPIR used by the decision path:

FPIR(τ) = 1 − (1 − FMR(τ))^N

Even a small pairwise FMR accumulates as the number of searchable entries rises. A threshold cannot therefore be a model constant. It depends on:

• the model and query cohort that were measured;
• the number of comparisons actually made; and
• the FPIR budget the review system can tolerate.

Veridion inverts the target FPIR at the effective search count, then finds the lowest cosine whose empirical impostor rate satisfies that pairwise requirement.

required FMR = 1 − (1 − FPIR_target)^(1 / N_effective)

The cut-point is an output of model × cohort × search scale × review budget, not a number copied into deployment configuration.

3. Do not extrapolate a tail the corpus cannot measure

A low false-match claim requires enough impostor comparisons. Even when zero of P impostor pairs cross a threshold, the approximate 95% upper bound is still 3/P. Veridion uses this rule of three as the empirical curve’s measurement floor.

When the required FMR falls below that floor, the system does not fit a Gaussian tail or extend the last observed point. Instead it:

• retains the similarity and ranking;
• leaves the user-facing verdict and threshold empty;
• records the comparison count and reason for withholding; and
• routes the case to second-stage verification or human review.

A missing or corrupt curve does not trigger a fallback to a fixed cut-off. In particular, a missing cropped-query curve cannot silently fall back to the easier whole-frame cohort. Refusing to attach a small number to an unmeasurable false-positive rate is intended behaviour.

4. Calibrate whole frames and crops separately

Cropping can raise genuine similarity by making a character larger, but it also discards background and contextual cues. If that context kept a character away from castmates in embedding space, impostor similarity rises too.

Query path	Calibration cohort	Effective comparisons
Single region	Whole-frame impostor curve	N
Multiple regions	Cropped-query impostor curve	regions × N

The maximum from several regions is not the result of one N-entry search. It is the maximum over R × N comparisons, so the decision path must use the same comparison load. Regions create more opportunities to recover a true character and more opportunities to encounter an accidental maximum.

The cropped curve is designed around held-out source regions searched against the actual index. If one composite contains characters A and B, both identities are removed from its impostor set. Getting that exclusion wrong would put a genuine near-match into the false-match tail and manufacture a score close to cosine one.

5. Finding two characters required both cropping and a fair merge

A model trained around one character per image can turn two characters into a blended embedding that belongs to neither identity. Veridion embeds and searches each detected region independently. If no valid region is found, the request falls back to the whole-frame path rather than returning an empty result.

The first max-fusion rule pooled every regional hit and sorted them again by similarity. It failed under unequal gallery depth: several references for one identity could occupy every visible slot, even when another region’s top hit correctly identified the second character. The replacement interleaves each region’s first result, then each region’s second result, and so on.

The repository records the following historical evaluation on a fixed 239-image gallery, 240 held-out two-character composites and 480 constituents. The table ranks deduplicated character identities, not individual endpoint rows.

Arm	R@1	R@2	R@5	R@10	Both characters in top 5
Cropping off	47.5%	56.0%	66.2%	75.6%	32.9%
multi_only with cropping	49.0%	86.2%	94.6%	96.9%	89.6%

The recorded paired R@2 difference is +30.2 points, with a constituent-level bootstrap 95% interval of [+25.8, +34.4]. Constituents share source images and characters, so they are not fully independent. Synthetic composites also omit the shared lighting, occlusion, speech bubbles and small figures found in real webtoon panels. The direction supports the method; the magnitude must not be transferred to real-panel performance.

6. Preserve failed experiments as reliability states

A high maximum alone can confirm a castmate from the same work or remain confidently wrong after stylisation. In one historical repository experiment, stencil transformation reduced character top-1 from 100.0% to 57.1% while cosine values remained ordinary-looking. A simple fusion of visual traits into identity scoring also regressed performance.

The design did not hide those results or merely raise the threshold.

• Different identities sharing a high band inside one region are marked ambiguous.
• Stylisation and out-of-population inputs become reliability states separate from score.
• Visual traits are removed from automatic verdict scoring and retained as explanations for review.
• Identity-level deduplication prevents one character’s many references from filling top-k.

A retrieval interface that displays the nearest row is different from an identification system that asks whether its evidence discriminated one identity.

7. Turn multimodal output into reviewable evidence

Veridion does not treat an image score as a copyright conclusion. Near duplication, character identity, Korean verbatim and semantic similarity, narrative, speech style and persona remain separate evidence channels.

The LLM cannot write free-form Turtle into the ontology. It emits a fixed JSON contract; the server constructs only allowlisted RDF triples. Model-derived claims are isolated by extractor in named graphs and carry PROV-O provenance, so a reviewer can trace the input and analysis path and withdraw one generated batch without damaging the authorial record.

The product therefore connects evidence rather than returning a binary “infringing/not infringing” label:

1. which source and rights assertion were compared;
2. which detector produced each candidate;
3. the scope and reliability state of every signal; and
4. what a human reviewer still needs to establish.

8. Reproducibility and claim boundary

This record was prepared by reviewing a retained source snapshot dated 10 August 2026. The implementation and evaluation harnesses remain, but the input corpus, vector index, curve artifacts and raw result files were not included.

Item	Retained state
Decision, retrieval and ontology implementation	Structure and contracts inspectable in source
Model-selection, open-set and crop evaluation harnesses	Inspectable in source
Input images, vector index and curve JSON	Absent from the retained source snapshot
Performance figures in this record	Historical repository-recorded measurements, not rerun while preparing this record
Current operating scale or performance	Not claimed

The source supports inspection of the algorithms and decision structure. It is not a self-contained benchmark release capable of bitwise reproduction of the tables. Visual similarity is not copyright infringement either: protectable expression, access or derivation, context of use and applicable exceptions remain human questions.

The next valid evaluation should preserve a releasable frozen subset together with its curve artifacts, split calibration from test identities, and include Korean webtoons and real multi-character panels. It should report whole-frame and cropped FPIR–TPIR curves, identity-cluster bootstrap intervals, latency and reviewer workload.

Until then, Veridion’s strongest claim is not an automatic accuracy number. It is that when scale or input shape changes the meaning of a score, the system changes the evidence required for a verdict—and stops when that evidence cannot be measured.