A search result showing that two images look alike does not finish a review. Someone still has to distinguish a cropped reuse of an original file from a new drawing of the same character. Similar sentences or character settings raise different questions again. I built Veridion to collect these signals, connect their sources and help a person compare the evidence. I was the sole contributor across requirements, architecture, backend, frontend and ontology implementation.
The part that changed most during development was the conclusion attached to a match. Models could retrieve candidates, but deciding when their output deserved confidence required a separate design. Finding a resemblance and deciding what that resemblance supports are different engineering problems.
Separate file reuse from character identity
I first separated the visual questions. SSCD, a near-duplicate detection model, looks for reuse of an image after transformations such as compression or cropping. CCIP, a character embedding model, looks for the same character across different poses and drawing styles. A redraw can be far from the original file while still depicting the same character; characters from the same series can also share a visual style without being the same person.
Text follows the same principle. Verbatim overlap and semantic similarity remain separate, alongside evidence about narrative, speech style and character settings. Combining their scores into a single apparent probability of infringement would hide what each signal means. I kept the question and limitations of each channel visible when assembling the review material.
A growing catalogue changed the meaning of a score
The initial fixed similarity threshold left out search scale. A false match that is rare when comparing a single pair becomes more likely when the same query is compared with many candidates. A threshold chosen on a small sample could not simply be carried over to the whole catalogue.
I changed the decision threshold from a model constant into a result of operating conditions. It uses the measured similarity distribution between different characters, the actual comparison count and the false-positive budget for review. The relevant quantity is FPIR: the probability that at least one wrong candidate passes during a search. The system works backwards from that budget to a threshold appropriate for the current search size.
That calculation also needs a stopping point. When the requested false-match rate is below what the measured sample can support, Veridion does not extend a fitted curve to invent an answer. It returns the ranking and similarity, leaves the verdict unset, and records the comparison count and reason for withholding it. Missing calibration data also causes withholding. Candidates outside the measured population are flagged separately, making the calibration’s scope visible. Retrieval remains available while the output distinguishes conditions that require withholding from those that need further validation.
Finding more characters introduced new false matches
A single embedding of a group image can land between the characters it contains. I added region detection so each character region could be searched separately, but detecting regions was only part of the solution. The initial merge pooled their matches and sorted by similarity. Characters with many reference images then filled the shortlist, pushing a correct match from another region out of view.
I changed the merge to interleave candidates by their rank within each region. The best candidate from each region gets a place before later candidates from those regions. Evaluation also distinguishes multiple references of one character from genuinely different character identities. This made the result list serve the review task: letting a person inspect the characters found across an image together.
Region search also required new calibration. Each region adds comparisons, and removing the background can increase similarity to wrong characters as well as correct ones. I first corrected the comparison count to include the number of searched regions, then measured a separate false-match distribution for cropped queries. Reusing the whole-frame calibration was insufficient. If the cropped-query calibration is missing, the system withholds its verdict instead of substituting the whole-frame threshold.
A fix for one image failed on the wider set
Heavily simplified artwork exposed another failure. Stencil-like transformations removed colour and shading that helped distinguish identities. The model returned a different character from the same series, while its similarity score still looked ordinary. The historical experiments documented a substantial gap between identification on original and stylised inputs.
I tested colour changes and grayscale conversion at query time. A particular failure image improved, but broader evaluation showed that these transformations could damage ordinary colour inputs or fail to repair the stylised cases. Adding similarly transformed references to the gallery did not recover the missing identity information either. Improvement on one example was not enough to justify adding preprocessing to every query.
I also tested mixing visual attributes such as hair, eyes and clothing into the retrieval score. Characters from the same series can share those attributes, and simple fusion made some correct rankings worse. I changed their role: retain the original retrieval and report which candidate the attributes support, including disagreements between the channels. An auxiliary candidate order can be shown without using it to promote the automatic verdict.
These failures led to an explicit reliability state alongside the score. Different characters crowded into a similar score range, strong stylisation and inputs outside the measured population should remain visible conditions. The reason a person needs to inspect a result became part of the result itself.
Evidence needs a way to be withdrawn
A match without provenance leaves the reviewer to find the original and its associated rights claim again. I connected images, text, characters, works and analysis sources in a graph. The vector index handles similarity search; the knowledge graph handles structural questions about where a claim came from and how it relates to other material.
Generated claims introduce another boundary. A relationship can be syntactically valid and still be false. The LLM therefore submits candidate claims through a fixed JSON structure, and the server constructs only permitted graph relationships. This constrains the shape of a claim; it does not establish its truth.
Model-derived claims remain separate from curated material. Each extractor writes into a distinct graph with provenance, so its claims can be withdrawn together if the extraction proves unreliable. That is the purpose of RDF named graphs and PROV-O provenance here. A model change should not leave the system unable to explain or undo what an earlier model asserted.
Implementation and validation scope
The result is a review system connecting retrieval, calibration, withholding and evidence inspection. A reviewer can inspect which region was searched, where candidates remain ambiguous and where the supporting material came from. My work was to define those decision boundaries and carry them through the response contracts and interface.
Improvements on synthetic multi-character images do not establish performance across real webtoon panels. Further evaluation needs frozen input and calibration artifacts, followed by measurement of false matches and review workload on the intended inputs.
Research record on search scale, region retrieval and withholding