Hook, Line, and Confidence: When Humans Outthink the Phish Bot

Phishing emails do not need to be brilliant. They only need to be plausible at the wrong moment.

A message about a failed payment, a suspended account, or an urgent verification request arrives while someone is clearing a crowded inbox. The user is not solving a formal classification task. They are deciding whether a sentence feels wrong enough to interrupt their day. That is why phishing defense is not only a machine-learning problem. It is a judgment problem disguised as an email problem.

The paper Evaluating Human and Machine Confidence in Phishing Email Detection: A Comparative Study looks at this judgment problem from a useful angle: not simply whether humans or models classify phishing emails correctly, but how their confidence and linguistic cues differ when they make the decision.¹ This is a small study, and it should not be mistaken for a production benchmark. But it asks a practical question that many enterprise security tools prefer to tiptoe around: when a model and a person both say “this looks suspicious,” do they mean the same thing?

The short answer is: not quite. Tree-based models can roughly match human majority judgments in this controlled setup. Humans, however, show more stable confidence and draw on a broader set of linguistic cues. Meanwhile, old-fashioned TF-IDF — the supposedly unfashionable bag-of-words cousin at the AI family dinner — often beats sentence embeddings for phishing-class F1 in this experiment.

That does not mean enterprises should abandon embeddings and return to keyword rules. It means the real design problem is not “human versus model.” The real problem is confidence-and-cue alignment: building phishing systems where machines, users, and security teams understand why a warning appears, how strongly to trust it, and when uncertainty should trigger escalation rather than automation theater.

The mechanism is not detection; it is calibrated suspicion

Most discussions of phishing AI begin with accuracy. That is understandable. Accuracy is easy to report, easy to rank, and easy to put in a vendor slide. It is also incomplete.

A phishing detection system has at least three layers of practical value:

The paper’s contribution is strongest at the second and third layers. It does report model performance, but the article becomes more interesting when we stop reading it as a mini-leaderboard and start reading it as a study of suspicion.

Suspicion is not binary. A user may correctly identify a phishing message while feeling only moderately confident. A model may be wrong while producing a high probability. A security team may receive thousands of alerts where the practical cost is not merely false positives or false negatives, but badly calibrated urgency.

In phishing defense, confidence is not decorative metadata. It changes behavior. A low-confidence model prediction might deserve human review. A high-confidence warning might justify stronger intervention. A user-facing alert with no explanation may produce fatigue. A user-facing alert with an intelligible cue — “unusual payment urgency,” “credential request,” “suspicious account language” — may teach future detection.

The paper therefore matters less because it says “humans are better” or “machines are better.” It does not really say either in a broad sense. It matters because it shows that humans and interpretable models can arrive at similar labels through different judgment patterns.

That difference is where product design begins.

What the study actually does

The authors train three interpretable machine-learning models: Logistic Regression, Decision Tree, and Random Forest. They test each model with two text representations: TF-IDF and Sentence-BERT embeddings. The point is not to beat the best phishing detector in the literature. The point is to compare transparent model behavior with human annotation behavior under the same input conditions.

For model training and evaluation, the authors use the Enron email dataset with binary phishing or legitimate labels. For the human-machine comparison, they generate a separate controlled test set using ChatGPT: 10 phishing emails and 10 non-phishing emails. Human participants annotate the same 20 emails, assigning a phishing/non-phishing label, a confidence score from 0 to 100%, linguistic cues that influenced the decision, and the emotion evoked by the email.

That setup has an obvious trade-off. The controlled test set makes the comparison cleaner. Humans and models face the same examples, so differences in label, confidence, and cue usage are not caused by different email samples. But the same design also limits generalization. Twenty synthetic emails are not the same as a messy corporate inbox filled with forwarded threads, broken formatting, regional writing patterns, attachments, embedded links, vendor notifications, and real attacker adaptation.

So the study should be read as a controlled comparison of reasoning patterns, not as a claim that a specific model configuration is ready for deployment. That distinction matters. A leaderboard asks, “Which model wins?” A cognition-aware security study asks, “What kind of judgment is each system making?”

The second question is more useful.

The “old” feature representation does surprisingly well

The first quantitative result is simple enough to state: TF-IDF generally performs better than embeddings for the phishing class in this setup, especially for Logistic Regression and Decision Tree.

There are two useful interpretations here.

First, phishing often leaves lexical fingerprints. Words and phrases around accounts, payments, verification, urgency, links, and support requests can still carry strong discriminatory value. TF-IDF is crude compared with dense semantic embeddings, but crude is not the same as useless. In some security problems, the suspicious object is not hidden in deep semantic nuance. It is sitting in repeated operational vocabulary with a fake sense of urgency.

Second, embeddings are not automatically better just because they are more modern. Sentence-BERT compresses an email into a dense semantic vector, which can help with broader meaning. But dense representation can also dilute specific lexical triggers that matter in phishing detection. A phishing message is often built from a small set of practical manipulations: verify this, update that, click here, act quickly, payment failed, account suspended. Semantic smoothness can be helpful, but the attacker’s craft often remains stubbornly textual.

This does not make TF-IDF the winner for all phishing detection. The study uses interpretable baseline models, a specific preprocessing pipeline, and a small synthetic test set for the human comparison. In real systems, modern models may perform better when headers, URLs, sender metadata, attachments, layout, and cross-email behavior are included. But the paper still gives a useful warning: do not assume that the newest representation captures the operational signal best.

In cybersecurity, sometimes the boring feature is boring because it works.

Humans and tree models can look similar at the label level

The paper then compares model outputs with human majority-vote labels on the same 20 generated emails. The authors report that humans performed equivalently to Decision Tree and Random Forest in the label comparison, while Logistic Regression predicted the most non-phishing emails.

The exact practical reading should be cautious. The figures mainly show label distributions and human/model comparisons on a small balanced set. This is not enough to claim universal human superiority or universal model equivalence. What it does support is narrower and more useful: in this controlled task, interpretable tree-based models and human majority judgment can land in broadly similar territory at the classification layer.

That is already interesting, because it shifts attention away from raw correctness. If the labels are similar, the next question becomes: what differs underneath?

The answer is confidence and cue usage.

This is the part security vendors often flatten. A dashboard may show “suspicious” or “safe.” A user training module may show “you clicked wrong” or “you clicked right.” A model evaluation report may show F1. But in real deployment, two systems with the same label accuracy can behave very differently.

One system may be confidently wrong. Another may be cautiously right. A third may be correct but unable to explain itself. A fourth may produce explanations that look plausible but do not match the actual decision process. Congratulations, we have built a compliance-friendly fog machine.

The paper’s more useful insight is that phishing detection needs to be evaluated as a judgment pipeline, not a label generator.

Confidence is where the model-human gap becomes visible

The confidence comparison is the paper’s central mechanism.

The authors use model class probabilities from predict_proba() and scale them to a 0-100 confidence range. Human participants self-report confidence on the same 0-100 scale. In the resulting comparison, humans show relatively stable average confidence, usually around 60-80%. The machine models, especially tree-based models, show sharper swings across emails.

This should not be overread as “humans are perfectly calibrated.” The study does not prove that. Human confidence is self-reported, subjective, and collected in an online survey. People can be confidently wrong, politely unsure, or strategically moderate. A 70% confidence rating from one participant is not necessarily equivalent to 70% from another participant.

Still, the pattern matters. Human confidence is steadier. Model confidence is more volatile. In an enterprise workflow, volatility is not automatically bad. A model should be more confident on easy cases and less confident on ambiguous ones. The problem is whether those confidence swings map to actual reliability and whether users understand what to do with them.

A phishing tool that outputs probability without calibration is like a smoke alarm that sometimes whispers during a fire and sometimes screams when toast is involved. Technically informative. Operationally exhausting.

The business question is therefore not “should we show confidence?” The question is “what action should confidence control?”

A practical phishing workflow could treat confidence as a routing signal:

This is where the paper’s mechanism becomes business-relevant. The point is not that humans should manually inspect every suspicious email. That would be an excellent way to create a security department made entirely of exhausted people and unread tickets. The point is that model confidence should help allocate human attention intelligently.

Cue usage explains why similar labels can still mean different reasoning

The study also compares the words humans and models identify as important cues. For TF-IDF-based models, important words are extracted through feature importance. For embedding-based models, per-word importance is not computed because dense embedding dimensions do not map cleanly back to individual words or n-grams.

This is an implementation detail, but it has a larger interpretability lesson. Sparse lexical models are easier to inspect because each feature corresponds to something humans recognize. Dense embeddings may perform well, but their internal dimensions are less directly explainable at the word level without additional methods.

The cue tables show that humans used a broader vocabulary when judging phishing and non-phishing emails. For phishing emails, model-selected words include repeated terms such as “account,” “desk,” “bank,” “support,” “revised,” “request,” and “life.” Human-selected words include “email,” “payment,” “verify,” “update,” “help,” “hours,” “funds,” and “link.” For non-phishing emails, model words again include repeated “account” entries along with “hello,” “schedule,” “attached,” “company,” and “employee,” while humans mention words such as “contact,” “password,” “office,” “account,” “schedule,” “payment,” and “process.”

This is not a perfect cue analysis. Some words appear in both suspicious and legitimate contexts. Some repeated terms may reflect the structure of the generated emails. The paper’s cue extraction is also lexical, not a full explanation of human cognition. Still, the contrast is useful.

Models optimize for separability. Humans judge plausibility.

A model may learn that “account” is frequently useful. A human may notice that “verify your payment information within 24 hours” combines institutional authority, financial urgency, and a call to action. That is not just a word list. It is a social script. Phishing works because attackers write messages that imitate institutional routines while bending them toward urgency and credential extraction.

This is the hidden challenge for explainable phishing AI. A system that says “important word: account” is not yet explaining the risk in a way that helps a user. A system that says “this email combines account-verification language with urgent payment consequences and an external action request” is much closer to useful explanation.

The paper does not build that system. But it points toward it.

The evidence blocks are doing different jobs

Not every result in the paper should be weighted the same way. Some pieces are main evidence. Others are implementation details or exploratory extensions. Treating them equally would make the article sound more certain than the study allows.

This distinction is important because the paper is most persuasive when used to reason about system design, not demographic prediction. The age result is interesting. The language result is interesting. But the central business lesson sits elsewhere: confidence and explanation should be treated as first-class components of phishing defense.

The demographic result is useful, but easy to misuse

The paper reports average accuracy by age group: 18-25 at 71%, 26-35 at 68%, 36-45 at 78%, and 45+ at 74%. It also reports that native and non-native English speakers were both around 71% accuracy on average, though English speakers showed more variability, including some individuals reaching 95%.

There is a tempting but lazy interpretation: older workers are better at phishing detection; language background does not matter; adjust training accordingly. That would be tidy. It would also be too confident.

A better interpretation is that experience, attentiveness, and exposure may matter more than simplistic assumptions about language proficiency. Mid-career and older participants may have seen more institutional email patterns and more suspicious messages. Non-native speakers may not be uniformly disadvantaged if the phishing cues are operationally clear enough. Native speakers may include both highly skilled and surprisingly vulnerable users. Human beings, tragically, continue to resist being compressed into clean HR segmentation variables.

For business use, the demographic result should inform training design lightly, not determine policy heavily. It suggests that security education should not assume younger users are naturally safer because they are “digital natives,” and it should not assume non-native English speakers are automatically weaker detectors. Better training should focus on cue recognition, contextual plausibility, and confidence discipline across groups.

The more actionable segmentation may not be age or language. It may be behavior: who clicks under urgency, who ignores warnings, who over-trusts familiar brands, who reports suspicious messages, and who changes behavior after feedback.

That requires real operational data, not only survey labels.

What this means for enterprise phishing defense

The practical lesson is not “replace the employee with a model.” It is also not “trust human intuition because humans are magical contextual creatures.” Please, no shrine to inbox wisdom.

The better interpretation is that phishing defense should be designed as a hybrid judgment system. Models are fast, scalable, and consistent. Humans are contextual, cautious, and able to interpret social plausibility. Security teams need both, but not in a vague “human-in-the-loop” slogan. They need routing logic, explanation design, and feedback loops.

A useful enterprise architecture would separate four functions.

First, automated triage should handle obvious cases. High-confidence phishing with clear lexical and metadata cues can be blocked or quarantined. Low-risk messages can pass with minimal friction. The system should not force every user to become a part-time threat analyst.

Second, ambiguous messages should receive calibrated warnings. The warning should not merely say “This may be phishing.” It should identify the cue pattern: urgency, account verification, payment pressure, external link, mismatched sender identity, unusual attachment, or abnormal language relative to the sender’s history. The exact cues would require stronger real-world systems than this paper tests, but the design principle follows from the cue analysis.

Third, human feedback should be captured structurally. When users report messages, override warnings, or hesitate before clicking, those actions can become signals for system improvement. The paper’s questionnaire asks humans for labels, confidence, cues, and emotion. A deployed system could collect lighter-weight equivalents: report reason, perceived suspicious cue, warning acceptance, and post-warning action.

Fourth, security training should target missed cues, not generic awareness. “Be careful online” is not training. It is a fortune cookie with liability insurance. Training should show users specific cue combinations they failed to notice: urgency plus credential request, payment language plus external link, authority language plus unusual sender behavior, or plausible internal tone plus abnormal workflow.

The paper does not validate this full architecture. Cognaptus infers it from the study’s comparison of labels, confidence, and cues. The direct evidence is controlled and narrow. The business implication is broader but conditional: if confidence and cue usage differ between humans and models, then enterprise systems should use those differences to allocate work.

What remains uncertain before deployment

The study’s limitations are not minor footnotes; they define how far the result can travel.

The human annotation test set contains 20 ChatGPT-generated emails, evenly split between phishing and non-phishing. That balance is useful for controlled comparison, but real inboxes are not balanced. In production, legitimate emails vastly outnumber phishing emails, and attackers adapt to filters, organizations, seasons, and current events. A model that behaves well on a balanced synthetic set may behave differently under real base rates.

The study also uses interpretable baseline models rather than state-of-the-art transformer systems. This is appropriate for the paper’s cognitive comparison because interpretable features can be inspected. But it means the results should not be read as a final statement about the best possible machine detector.

Human confidence is self-reported. That matters. In a survey, participants have time to read, reflect, and move a confidence slider. In a workplace inbox, the same person may be multitasking, tired, interrupted, or using a mobile screen while half-listening to a meeting. Confidence in a survey is not the same as hesitation before a click.

The cue analysis is also limited. Word-level cues are informative, especially for TF-IDF models, but phishing often depends on relationships among cues: sender identity, institutional context, link target, requested action, timing, prior thread history, and interface presentation. A word list is a start, not a cognitive map.

Finally, the language-background result is too small and context-specific to support broad conclusions. It is useful as a warning against crude assumptions, not as a universal claim that language background never matters.

These boundaries do not weaken the article’s main business lesson. They sharpen it. The paper should influence how teams think about human-AI phishing workflows, not serve as a plug-and-play procurement benchmark.

The real KPI is not just fewer clicks

Security teams often measure phishing defense by click rate, report rate, false positive rate, and detection accuracy. Those metrics matter. But this paper suggests additional metrics that may be just as important for human-AI collaboration.

A phishing defense program should ask:

• Are model confidence scores calibrated against actual outcomes?
• Do warnings explain cue patterns that users can understand?
• Do users become better at identifying suspicious cue combinations over time?
• Do ambiguous emails route to humans rather than pretending automation is certain?
• Do human reports reveal cues the model underweights?
• Do different user groups fail for different reasons, or merely at different rates?

This is a different way to evaluate phishing AI. It is less glamorous than announcing a higher accuracy score. It is also more operationally honest.

The model does not need to “think like a human” in some grand philosophical sense. It needs to support the human at the moment of decision. That means knowing when to block, when to warn, when to explain, and when to admit uncertainty.

A classifier that is right but inscrutable is useful for filtering. A classifier that is right, calibrated, and cue-aware is useful for collaboration.

That is the difference between a phishing detector and a phishing defense system.

Conclusion: the best system teaches suspicion without spreading paranoia

The paper’s quiet message is that phishing detection is not only about catching bad emails. It is about shaping better judgment under uncertainty.

Humans in this study do not simply lose to machines. Machines do not simply outperform humans. Tree-based models can produce comparable labels, TF-IDF can outperform embeddings for phishing-class F1 in this setup, and humans can show steadier confidence and broader cue usage. The interesting result is the mismatch beneath the label.

For enterprises, that mismatch is an opportunity. Use models for scale. Use confidence for routing. Use explanations for training. Use human feedback to find missed cues. And above all, stop treating phishing defense as a binary button labeled “AI solved it.”

The inbox is not a benchmark. It is a messy interface where people decide whether to trust a message. Any AI system that ignores that human moment is not defending the organization. It is just sorting text very quickly.

And as every security team eventually learns, very quickly is not the same as very wisely.

Cognaptus: Automate the Present, Incubate the Future.