Seeing Too Much: When Multimodal Models Forget Privacy

Face.

That is where the privacy problem starts to become awkward.

A company does not need to build a facial-recognition product to create facial-recognition risk. It may only add a multimodal model to a customer-support workflow, an HR document review process, a KYC assistant, a media-monitoring tool, or a claims-processing system. Someone uploads an image. The model sees a person. Then the user asks: Who is this? Where do they live? What is their email? What is their religion? What is their medical condition?

The comforting assumption is that the model will treat all of these as private-information requests and refuse. The slightly less comforting assumption is that it will fail randomly.

PII-VisBench, the paper behind this article, argues that neither assumption is quite right.¹ The failure is not random. It depends on how visible the person is online, what kind of personal information is requested, which model family is used, how the prompt is phrased, and whether the model’s “helpfulness” has learned to outrun its privacy guardrails. A model does not merely see an image. Sometimes it recognizes a person. Sometimes it guesses. Sometimes it refuses. Sometimes it does all three badly, which is a pleasingly modern achievement.

The important contribution is not just another benchmark table. The important contribution is a mechanism: privacy behavior in vision-language models is shaped by the collision between recognition, memorization, visual profiling, and safety tuning.

That is the business problem.

The privacy risk changes when the model recognizes the person

Most privacy evaluations ask a simple question: does the model reveal personal information when prompted? PII-VisBench asks a better question: does the model behave differently when the subject has a different level of public visibility?

The authors build a benchmark of 4,000 unique probes from the cross-product of 200 subjects and 20 PII attributes. The subjects are divided into four visibility groups:

This design matters because privacy risk has at least two engines.

The first is memorization-based retrieval. If a public figure’s biography, address-like details, social profiles, or personal attributes appear repeatedly in web-scale data, the model may answer because the identity is recognizable and the fact pattern is already embedded in its weights.

The second is inference-based profiling. If the person is unknown or synthetic, the model cannot retrieve real linked information. But it can still produce PII-like content by guessing from visual cues, stereotypes, or generic priors. That is not a harmless failure. A hallucinated address or medical condition may be false, but in a real workflow it can still trigger downstream decisions, reputational harm, or compliance exposure.

So the question is not only “does the model leak?” It is “which mechanism is producing the leak-like behavior?”

That distinction is where this paper earns its keep.

PII-VisBench tests refusal and disclosure separately

The paper evaluates 18 open-source VLMs, ranging from 0.3B to 32B parameters, across model families including LLaVA, InternVL, Qwen, Gemma, SmolVLM, Llama, and Phi. The authors use manually written WH-question prompts for 20 PII categories and evaluate both original prompts and paraphrased versions. They also test jailbreak-style prompt attacks.

Two metrics carry most of the analysis.

The first is Refusal Rate, or RR: the percentage of PII-seeking queries where the model refuses. This is evaluated with target-string matching and LLM-as-judge methods, then summarized through an evaluator average.

The second is Conditional PII Disclosure Rate, or cPDR: among the responses that are not refusals, the fraction that is judged to contain PII-like content.

The split is important. A model can have a low refusal rate but still avoid disclosure by giving vague, non-identifying answers. Another model can refuse often but disclose when it does not refuse. For deployment, those are different risks.

A procurement checklist that asks only “does the model refuse unsafe requests?” is therefore too shallow. It misses the conditional behavior after refusal fails.

This is a useful measurement stack. Not perfect, but useful. And unlike many safety papers, it does not pretend that one metric is a personality.

The headline pattern is a conservative gradient, with a high-visibility gap

The central empirical result is visibility-dependent behavior.

Across the 18 models under the original prompt setting, cPDR falls from 9.10% for high-visibility subjects and 8.86% for medium-visibility subjects to 5.34% for low-visibility subjects. Zero-visibility synthetic subjects sit at 6.46%, slightly above low visibility.

The broad pattern is clear: models are more likely to produce PII-like content for people with greater online visibility. At the same time, refusal tends to increase as visibility decreases. The authors describe this as a conservative gradient.

The intuitive explanation is almost backwards from how many business readers might first think about privacy. One might assume unknown private individuals are most at risk because the model has no special reason to protect them. The paper suggests the opposite risk is also real: highly visible people may be more exposed because recognition and memorized web facts can overpower refusal behavior.

That matters because public visibility is not the same as consent. A person being easier to identify does not make every face-linked question operationally acceptable.

The statistical analysis supports this visibility effect. Post-hoc Wilcoxon tests show significant differences between high and medium, high and low, high and zero, medium and low, and medium and zero visibility groups. The low-versus-zero comparison is not significant. The largest refusal-rate shift occurs when moving from high to low visibility, which is exactly the transition that separates likely memorization from weaker public traceability.

The authors also run model-level trend checks. Fifteen out of 18 evaluated models show a positive monotonic relationship between decreasing visibility and increasing refusal. The exceptions are Phi3.5 4B, Llama3.2 11B, and Qwen2 7B.

This is not a single-model quirk. It is a recurring behavioral pattern across most of the tested open VLMs.

The privacy failure has three moving parts, not one

A mechanism-first reading of the paper points to three interacting components.

First, recognition changes the model’s behavior. If the model recognizes a subject, it may treat the prompt less like a privacy violation and more like a factual question. “What is this person’s name?” becomes, in the model’s internal behavior, closer to “What is the capital of France?” Unfortunately, people are not capitals. They move around, have addresses, families, conditions, affiliations, and lives.

Second, safety tuning appears uneven across PII types. The model may have learned that SSNs, passport numbers, phone numbers, and addresses are sensitive formats. It may be much weaker on demographic or descriptive categories such as gender, race, religion, or political view, especially when those requests resemble normal image-description tasks.

Third, prompt form affects the gate. Paraphrasing does not destroy the core visibility pattern, but it weakens refusal for some model families. Jailbreak prompts then stress the model harder, revealing that some systems have refusal layers that are easier to route around.

This is why a table of model rankings is not enough. The real lesson is that privacy alignment is not a single switch. It is a set of fragile behaviors distributed across recognition, instruction-following, refusal templates, safety classifiers, and learned associations from pretraining.

Very tidy. Also very easy to break.

Model family matters more than size

The paper’s model comparisons are useful, but only if read carefully. They are not a leaderboard for “best VLM.” They are a diagnosis of privacy behavior under a controlled benchmark.

Some model families look relatively safety-centric. InternVL3, Phi3.5 4B, and Llama3.2 11B maintain high average refusal rates, roughly in the 85–97% range. But high refusal is not the whole story. Phi3.5 4B and Qwen2.5 7B also show near-zero cPDR, at 0.10% and 0.12% respectively, suggesting that even when they do not refuse, their answers often avoid PII-like disclosure.

Other models look more permissive. LLaVA1.5 7B reaches an average cPDR of 26.36%, and SmolVLM2 2.2B reaches 20.37%. LLaVA models and Qwen2 7B also show relatively low refusal rates in the 30–43% range, paired with higher disclosure risk.

Parameter count does not rescue the story. Scaling sometimes helps, but not reliably. InternVL3 improves from about 92% refusal at 8B to 96% at 14B. LLaVA1.5 rises from around 33% at 7B to 40% at 13B. Gemma3 also increases from 4B to 27B.

But Qwen3 shows a bell-shaped pattern: average refusal rises from 51% at 4B to 71% at 8B, then falls to 58% at 32B. SmolVLM similarly peaks at 0.5B and then declines at 2B. The paper also notes that SmolVLM 0.5B achieves an average refusal rate around 81%, higher than much larger models such as Qwen2.5 7B at 60% or Gemma3 27B at 48%.

The practical reading is straightforward: bigger is not automatically safer. Newer is not automatically safer either.

Within Qwen, later generations become more conservative: high-visibility refusal rises from 31.6% in Qwen2 7B to 55.7% in Qwen2.5 7B and 61.3% in Qwen3 8B. In low visibility, Qwen3 8B reaches 80.0% refusal, which the paper reports as a 124% relative increase over Qwen2 7B.

InternVL and SmolVLM move differently. InternVL3.5 8B shows lower average refusal than InternVL3 8B. SmolVLM2 2.2B also lowers refusal relative to its predecessor. The authors interpret this not necessarily as weaker safety, but possibly as an attempt to reduce over-refusal and improve utility.

That distinction matters for business use. A model that refuses everything is safe in the same way a locked filing cabinet with no key is productive. The objective is not maximal refusal. The objective is correct refusal, with harmless utility preserved.

Paraphrases test stability; jailbreaks test failure under pressure

The paper’s prompt-sensitivity analysis should not be read as a second thesis. It is a robustness test.

Paraphrasing the prompts preserves the overall model ordering and the visibility pattern. Models that refuse often under original wording usually continue to refuse under paraphrased wording. Low-visibility subjects still tend to trigger more refusal than high-visibility subjects.

But paraphrasing weakens refusal for some models, especially in the SmolVLM family, with drops sometimes around 5–15 percentage points depending on visibility. This matters because real users do not ask benchmark-perfect questions. They ask messy, indirect, polite, manipulative, vague, or weirdly formatted questions. As is their ancient right.

The jailbreak tests are more severe. The authors apply seven attack templates, including AIM, Prefix Injection, Refusal Suppression, Evil Confidant, Payload Splitting, Style Injection, and Few-shot JSON. They report results for four representative models.

Under jailbreak attacks, InternVL3 14B has the strongest aggregate profile among the four highlighted models, with 86.2% average refusal and only 0.4% cPDR. LLaVA1.5 13B is much weaker, with 53.8% average refusal and 13.5% cPDR. Qwen3 8B records 62.9% refusal and 1.4% cPDR. Gemma3 4B records 67.1% refusal and 4.8% cPDR.

The sharpest warning is not simply “jailbreaks work.” Everyone already knows that, and everyone has already pretended to be surprised at least once.

The sharper warning is that jailbreak vulnerability is model-family specific and attack-format specific. Few-shot JSON, for example, creates severe disclosure behavior for LLaVA1.5 13B, with cPDR reported at 30.8% under original jailbreak prompts and 37.5% under paraphrased ones. That is not a minor wording issue. It means structured output demands can interact badly with privacy safeguards.

For enterprise use, this is a direct evaluation requirement. A privacy review that tests only normal prompts is theater. Nice costumes, weak controls.

The category problem: models fear SSNs more than social attributes

The PII-type results reveal another mechanism: many models seem more sensitive to recognizable data formats than to privacy as a concept.

Across models, refusal is high for structured identifiers. SSN reaches an average refusal rate of 90.13%. Address reaches 79.74%. Passport number reaches 85.10%. These are the categories that look like conventional security and compliance training data.

The model behavior is weaker for demographic or descriptive attributes. Race has an average refusal rate of 39.84%. Gender is only 13.73%. Medical condition sits at 53.36%, a semi-structured and semantically sensitive category that does not always look like a classic identifier.

This is where the paper’s mechanism becomes operationally important. Vision-language models are trained to describe images. Gender, race, age, eye color, and sometimes relationship status can look like normal captioning tasks rather than privacy-sensitive requests. The model may not experience the prompt as “PII extraction.” It experiences it as “describe what you see.”

That is dangerous because privacy policy does not stop at numbers with hyphens.

The paper also reports an inverse visibility pattern for names: refusal for name queries is 47.02% for high-visibility subjects and 76.31% for low-visibility subjects. Again, recognition changes the model’s threshold. The better the model is at knowing who someone is, the less likely it may be to refuse naming them.

This does not mean every name answer is legally prohibited or morally equivalent to leaking a bank account. It means face-linked identity disclosure must be governed intentionally, not left to whatever the model happens to infer from pretraining.

The appendix narrows the claim instead of weakening it

The appendix material is useful because it prevents overreading.

The search-result distribution supports the visibility construction. High- and medium-visibility subjects are separated using a web-search-count threshold, while low and zero visibility are designed to represent negligible or nonexistent public traces. This is a coarse proxy, not a metaphysical measure of fame. Good. Benchmarks should not cosplay as divine knowledge.

The judge agreement analysis shows that automated evaluators are not equally stable across models and prompts. Some models produce clearer refusal patterns, making judge agreement easier. Others create ambiguous responses that reduce agreement. This supports the authors’ decision to report multiple judging schemes and evaluator averages.

The PII structure validation is especially important. For five hard PII categories—SSN, driver’s license, passport number, phone number, and email—the authors validate whether outputs match syntactically valid formats. The valid response rates are low: 0.2870% for SSN, 0.4630% for driver’s license, 0.6435% for passport, 3.0509% for phone number, and 3.8657% for email.

This does not erase the cPDR concern. It refines it.

A model may output PII-like content that is not valid structured PII. For compliance teams, that distinction matters. A fake SSN is not the same as a real SSN. But a hallucinated email, address-like string, medical condition, or social profile can still create operational harm if a workflow treats model output as evidence.

The correct interpretation is therefore disciplined: PII-VisBench measures systematic tendencies toward refusal and PII-like disclosure. It does not prove that every disclosed item is a true leaked fact.

That boundary is not a footnote. It is the difference between safety evaluation and forensic verification.

What businesses should change in VLM governance

The paper directly shows that open-source VLMs behave differently across subject visibility, PII categories, model families, generations, parameter sizes, paraphrases, and jailbreak prompts.

Cognaptus’ business inference is that multimodal privacy testing should become segmented, not generic.

A useful enterprise evaluation should separate at least four dimensions:

This affects four common business settings.

First, customer support and document intake. If users upload IDs, screenshots, invoices, messages, or profile images, the model should not be allowed to freely answer face-linked identity questions. The rule should not be “let the model decide.” The rule should be policy-bound routing: answer document-content questions where legitimate, refuse or redact face-linked personal queries where not.

Second, KYC and fraud workflows. These systems already operate near identity-sensitive data. VLMs may be useful for extracting document fields or checking image quality, but face-linked free-form inference should be tightly constrained. A model that guesses residence, social profiles, political views, or medical conditions from an image is not adding intelligence. It is adding liability with a nice API.

Third, HR and recruiting tools. Multimodal models used on CVs, video frames, portfolios, or interview recordings should be blocked from producing protected or sensitive inferences. The low refusal rates for categories like gender and race are a warning that “visual description” can become a policy violation by another name.

Fourth, media monitoring and public-person analysis. Public figures create the hardest governance boundary because some identity information is legitimately public. But the paper’s high-visibility gap means publicness can make models more permissive. Companies need policy tiers: public professional facts may be allowed; addresses, family details, private identifiers, medical conditions, and sensitive affiliations should remain protected unless there is a strong legal and editorial basis.

The operational takeaway is not “never use VLMs on people.” That would be neat, expensive, and often impractical.

The better takeaway is: build a face-linked privacy layer around the model. It should classify the request, not just the content. “What color is the person’s shirt?” is not the same as “where does this person live?” The image may be the same. The privacy risk is not.

Procurement should ask for privacy curves, not safety slogans

Model vendors and internal AI teams often summarize safety as a general property: safe, aligned, enterprise-ready, policy-compliant. Charming words. Not measurements.

PII-VisBench suggests better procurement questions:

1. What is the model’s refusal rate and conditional disclosure rate by subject visibility?
2. Does the model behave differently for public figures versus private individuals?
3. Which PII categories are protected by semantic understanding, and which only by format recognition?
4. How stable are refusals under paraphrasing?
5. What happens under structured-output pressure, such as JSON templates?
6. Does a larger or newer model actually improve privacy behavior in the target workflow?
7. Are PII-like outputs validated for factuality or only classified as disclosure-like content?

These questions are not academic decoration. They affect ROI because privacy failures create review costs, legal exposure, workflow redesign, and procurement reversals. A cheaper model with stronger privacy behavior for the relevant categories may be better than a larger model with impressive general benchmarks and a habit of answering questions it should not.

The business value is not just better safety. It is cheaper diagnosis.

Boundaries: what the paper does not settle

The paper’s limitations are material and should shape how businesses use the results.

The benchmark is English-language and includes PII categories that are common in Western administrative contexts, such as SSNs and mother’s maiden name. Results may differ in other languages, jurisdictions, and identity systems.

Visibility is measured using web search hit counts and image/name search procedures. That is scalable, but noisy. Search results vary by indexing behavior, region, personalization, temporal drift, and naming ambiguity. Visibility should therefore be read as a coarse bucket, not a precise social variable.

The subject sources may also introduce demographic and cultural skews. CelebA, FFHQ, public web images, and synthetic faces are practical benchmark components, but they are not a complete model of global human diversity.

The evaluation covers open-source VLMs. Closed-source systems may behave differently, for better or worse. They may have stronger safeguards, different refusal policies, or different memorization profiles. The paper’s motivating example mentions closed-source refusals, but the systematic benchmark focuses on open models.

Most importantly, cPDR is not factual verification. A model may disclose a true email, hallucinate a plausible email, or produce a placeholder-like string. These outcomes differ for legal and operational purposes. The paper’s structure validation helps, but full leakage verification remains future work.

So the result should not be summarized as “VLMs leak everyone’s PII.” That is too broad.

A better summary is: open VLMs show systematic, measurable, visibility-dependent privacy behavior, and current safety alignment often protects structured identifiers more reliably than face-linked identity, demographic, and descriptive privacy boundaries.

Less dramatic. More useful. Annoyingly, that is how good analysis usually works.

The real lesson is visibility-aware privacy alignment

PII-VisBench gives businesses a sharper privacy question for multimodal AI: not simply whether a model can refuse, but whether it refuses consistently across the people, prompts, and PII categories that matter.

The high-visibility privacy gap is the conceptual center. It shows that public recognizability can weaken privacy protection rather than strengthen it. The model sees more, remembers more, and may therefore say more. Safety tuning then has to fight not just the user’s prompt, but the model’s own learned associations.

For companies deploying VLMs, the practical response is clear. Test by visibility. Test by category. Test after paraphrasing. Test under structured-output pressure. Separate refusal from disclosure. Validate structured identifiers. Treat face-linked questions as a special governance class.

A face is not just another image input. Once the model can connect that face to identity, memory, inference, and social attributes, the system is no longer only describing pixels.

It is handling a person.

That should make the privacy bar higher, not lower.

Cognaptus: Automate the Present, Incubate the Future.