TL;DR for operators
Zheng and colleagues’ Less is More study1 shows that image search can benefit from two different levels of scrutiny: concise descriptions for retrieving candidates, followed by detailed visual verification of a shortlist. On CIRCO with ViT-L/14, adding a vision-language critic raises mAP@5 from 29.01 to 39.82, a 10.81-percentage-point improvement over concise retrieval alone.
The architecture separates two requirements that ordinary similarity scores can confuse: make the requested change and preserve everything that should stay the same. Yet the critic examines only the first 20 candidates. It cannot recover a target excluded earlier, and its visual judgments can introduce ranking errors.
For product-search teams, the evidence supports testing this two-stage design against simpler retrieval pipelines. The unresolved questions concern candidate coverage, false demotions, inference latency, and operating cost—not whether the reported benchmark improvements exist.
A better match can disappear before verification begins
A shopper finds a shirt with the right design but wants a different color. The desired product should retain its cut, pattern, and other relevant characteristics while satisfying the requested change.
Finding a visually similar shirt is insufficient. A search result might preserve the original color but change the design, or satisfy the color instruction while losing the features that motivated the search.
This is the problem of composed image retrieval: finding a target image from a reference image and a written modification instruction.
A common strategy is to generate a detailed description of the intended target and search for images matching that description. But extra details may refer to irrelevant background objects, incorrectly inferred attributes, or things the user specifically wants removed.
There is also a second problem. A candidate that looks broadly similar in an embedding space may violate a small but decisive instruction. Detecting that violation requires more detailed examination than the initial search provides.
The reported research addresses these problems separately. Its results also expose the consequences when either stage makes a mistake.
Concise descriptions improve retrieval by excluding distracting information
LM-CIR uses a training-free, two-stage architecture. Training-free here means no composed-image-retrieval-specific fine-tuning; the system still relies on pretrained models, prompting, and inference.
In its first stage, GPT-5.1 generates a concise description of the desired target image. The prompt guides the model through identifying reference objects, interpreting requested changes, inferring the target appearance, and retaining only necessary descriptive information.
The prompt discourages unsupported specificity, removed objects, and negative expressions. This matters because embedding-based retrieval may respond to concepts appearing in a caption even when those concepts describe something that should not appear in the target.
The system then combines two CLIP cosine-similarity signals: how well a candidate matches the generated caption, and how closely it resembles the reference image. The reference-similarity contribution receives a weight of 0.2 on CIRCO and CIRR, or 0.3 on FashionIQ.
The caption-only comparison provides evidence for this compression strategy. On CIRCO, with other retrieval components removed, concise GPT-5.1 captions achieve 25.28 mAP@5, compared with 24.05 for AutoCIR, 23.87 for OSrCIR, and 18.57 for CIReVL (Section 4.5, Table 4).
This comparison supports the value of the reported concise-caption approach under its evaluation conditions. It does not establish that shorter captions universally outperform longer ones, especially when fine-grained visual attributes distinguish otherwise similar products.
It also compares outputs from different caption-generation approaches, rather than isolating caption length as the only changing variable.
The central design principle is therefore selective description: retain the attributes needed for matching while suppressing information that could redirect retrieval toward the wrong image.
The critic checks what similarity scores leave unresolved
Concise descriptions help generate candidates, but they do not establish whether a particular candidate correctly implements the user’s instruction.
LM-CIR addresses this with a second stage. Qwen2.5-VL-32B-Instruct examines each of the top 20 retrieved images alongside the original reference and modification instruction.
The critic assigns two scores, each from 0 to 10:
- EDIT: Does the candidate satisfy the requested modification?
- PRESERVE: Does it retain reference-image attributes that should remain unchanged?
The final critic score weights EDIT at 0.7 and PRESERVE at 0.3. The system then reorders those 20 candidates, leaving the rest of the gallery ranking unchanged.
This distinction matters because a candidate can perform well on either criterion while failing the other.
The scoring-weight ablation makes the difference measurable. On CIRCO with ViT-L/14, PRESERVE-only scoring produces 30.97 mAP@5, EDIT-only scoring produces 37.67, and their weighted combination reaches 39.82 (Section 4.5, Table 3).
This is an ablation testing the scoring design, rather than a separate demonstration of deployment reliability. It shows that the two criteria contribute complementary information in the evaluated configuration.
The broader benchmark results are substantial. With ViT-G/14, full LM-CIR achieves 42.79 mAP@5 on CIRCO, versus 32.75 for the strongest reported comparator on that metric, IP-CIR—a 10.04-point advantage. On CIRR, Recall@1 reaches 47.47, versus IP-CIR’s 39.25 (Section 4.3, Table 1).
These are early-rank retrieval improvements: relevant images appear nearer the beginning of the result list. The published baseline comparisons were not all independently re-executed under identical caption-generation configurations, so they should be read as reported benchmark comparisons rather than fully controlled component experiments.
The shortlist determines what the critic can fix
The same architecture that improves rankings imposes a strict coverage limit.
In the reported CIRCO ablation with ViT-L/14, evaluating the top five candidates yields 32.82 mAP@5. Expanding the critic’s input to ten candidates increases that figure to 36.82, while examining 20 reaches 39.82 (Section 4.5, Table 3).
This sensitivity test shows why candidate-pool depth matters. More candidates provide the critic with additional opportunities to promote relevant images, but also require additional model evaluations.
Crucially, the critic cannot promote an image it never receives.
Section 4.6 reports a failure in which an ambiguous modification instruction leaves the ground-truth image beyond the first 50 retrieved results. Re-ranking the top 20 cannot address that failure, regardless of how accurately the critic evaluates the available candidates.
A second failure involves a blue Finland soccer T-shirt. The critic incorrectly interprets the relevant visual content and rejects a target that the first stage had already ranked correctly (Section 4.6, Figure 5).
These failures arise from different sources. The first is a candidate-coverage problem. The second is a critic-judgment problem.
The latter also appears in the critic-model ablation. Under the reported CIRCO ViT-L/14 configuration, Qwen2.5-VL-32B reaches 39.82 mAP@5, compared with 35.87 for Qwen2.5-7B and 21.84 for LLaVA-OneVision-7B (Section 4.5, Table 3). Model choice materially affects the outcome, although architecture, training, and perceptual capability vary together in this comparison.
A critic is therefore another fallible model component, not an independent guarantee of correctness.
Early-rank gains do not establish production superiority
FashionIQ provides a useful boundary to the results.
With ViT-G/14, critic re-ranking improves average Recall@10 from 44.01 to 45.75. But average Recall@50 remains 64.57 because reordering the top 20 does not change which images appear in the top 50.
IP-CIR achieves 45.74 average Recall@10 and 66.28 average Recall@50 in the same reported FashionIQ comparison. LM-CIR is marginally ahead at Recall@10 but behind at Recall@50 (Section 4.3, Table 2).
The method therefore does not dominate every benchmark metric. Its strongest evidence concerns improving the ordering of initially retrieved candidates, particularly at early ranks.
For an e-commerce search team, Cognaptus draws three evaluation priorities from this evidence.
First, measure whether the initial retrieval stage places relevant products inside the verification pool. Broader candidate retrieval may matter more than stronger verification when target images are routinely missing from the shortlist.
Second, track both successful promotions and false demotions. An additional critic should be assessed not only by aggregate ranking gains but also by how often it displaces results that were already correct, especially for logos, text, colors, and fine-grained product attributes.
Third, measure incremental inference cost and latency against the search experience actually being optimized. The paper uses up to 20 candidate-level critic evaluations per query, but it does not report end-to-end production latency, throughput, API expenditure, or operating cost. Its benchmark improvements alone cannot determine whether the extra processing is commercially worthwhile.
FashionIQ was also evaluated on validation data rather than an accessible official test server, and the study does not report confidence intervals or deployment-distribution tests. These limitations restrict how confidently the measured gains can be transferred to a live catalog.
Better retrieval requires both coverage and judgment
LM-CIR demonstrates that generating more descriptive text is not necessarily the best way to improve image retrieval. Concise, target-oriented descriptions can produce a stronger initial ranking, while a detailed critic can correct some errors that embedding similarity leaves unresolved.
Its most consequential architectural lesson is the separation of candidate discovery from compositional verification. The former determines which images are available for consideration; the latter determines how accurately those images are ordered.
For search-system designers, the research offers a credible reason to test that separation. It also specifies where evaluation must become more demanding: missing candidates, incorrect critic judgments, and the economic cost of examining a larger shortlist.
The critic can improve the ranking substantially. The result still depends on the images it is allowed to see—and on whether it understands what it sees.
Cognaptus: Automate the Present, Incubate the Future.
-
Yuqian Zheng and Hyoju Ahn and Hyeongjin Ahn and Eunil Park (2026). Less is More: Concise Retrieval and Critic-based Re-ranking for Composed Image Retrieval. arXiv:2610.27538. https://arxiv.org/abs/2610.27538 ↩︎