Who Owns the Words of AI-Assisted Support?

TL;DR for operators

An AI writing assistant can produce a supportive message that users rate highly without preserving their sense of having personally offered support. In PeerPen’s 15-participant study, people generally experienced writing their own response and selectively accepting AI revisions as more collaborative than proofreading an AI-generated draft.

The operational implication concerns who establishes communicative intent and controls the subsequent writing process, rather than merely who approves the completed text.

For platforms introducing AI into emotionally sensitive communication, consider user-first drafting, individually accepted revisions, graduated assistance, and clear rules for disclosing AI involvement. Evaluate perceived authorship and recipient trust alongside response quality and convenience.

These are design directions supported by exploratory user research, not demonstrated improvements in clinical safety, participation, or community trust.

A supportive message can still feel improperly authored

Consider someone reading a post from a stranger describing emotional distress. They want to respond but struggle to find appropriate words. A writing assistant offers two options: generate a complete supportive response for approval, or help improve a response the person has already written.

Both workflows could produce readable, empathetic messages. Yet they allocate responsibility for the communication differently. In one, the system proposes what to say; in the other, the person establishes what they want to communicate before the system helps refine the language.

This difference emerges in PeerPen: AI-Assisted Writing for Online Mental Health Peer Support, by Jiwon Kim and colleagues.¹ Participants generally described revising their own writing with AI as more collaborative than correcting a primarily AI-generated response, even though both approaches received favorable response-quality ratings.

The finding challenges a common product assumption: allowing a user to approve AI-generated text may not be sufficient to preserve their sense of authorship.

Authorship depends on who decides what to communicate

The study suggests that perceived authorship depends on the sequence of human and AI contributions.

When someone writes an initial message, they have already selected its purpose, decided which aspects of another person’s experience to acknowledge, and chosen what support they are prepared to offer. AI revisions can then improve the expression of those choices.

When AI generates the initial response, the human instead evaluates a message whose communicative direction has largely been established by the system. Editing that message may involve substantial effort without producing the same sense of ownership.

The authors develop this into an account of process-dependent authorship: ownership is shaped by who supplies communicative intent and exercises control over subsequent changes, rather than simply by the number of words each contributor produces.

That distinction has practical consequences. A person might accept several substantial revisions while retaining responsibility for the message’s purpose. Conversely, someone who lightly edits an AI-generated response may approve its content without feeling they personally composed it.

This is an interpretation of participants’ experiences, not an experimentally established formula for measuring authorship. It nevertheless identifies a design variable that conventional text-quality evaluation misses.

PeerPen makes generation and revision different interactions

PeerPen embeds writing assistance inside a Reddit-like environment. Its two principal features give users different ways to collaborate with the system.

The Improvement Feature keeps the user’s original text available for comparison. Suggestions are presented as separate changes rather than automatically replacing the complete response.

This provides granular revision control: the writer can adopt one proposed phrase while rejecting another that changes their intended meaning or personal voice.

The distinction is not merely cosmetic. Participants sometimes found AI suggestions generic, repetitive, slow, or stylistically inconsistent. Assistance could shift work from composing a response to correcting language that did not fit the writer’s intention.

Localized revisions also exposed participants to alternative supportive expressions, questions, and practical suggestions. However, seeing a change did not necessarily explain why it was preferable. The authors therefore recommend clearer explanations and progressively stronger assistance rather than assuming every user needs a complete draft.

PeerPen uses Generation and Improvement models adapted from Mistral-7B in the earlier LLUMI research. The present study investigates how people use the writing interface; it does not introduce or independently benchmark a new model-training method.

Favorable response ratings leave authorship unresolved

The researchers recruited 15 US-based participants with experience in online mental health communities. Participants completed approximately four to five writing tasks during remote sessions lasting about one hour, accompanied by think-aloud observations, interviews, and post-interaction surveys.

The surveys covered 17 GF-generated and 54 IF-revised responses. Table 3 reports the following mean ratings on five-point scales:

Participants assessed both modes favorably across these dimensions. Actionability showed the largest difference between the reported means, favoring IF by 0.42 points. Its standard deviations were 1.07 for GF and 0.87 for IF, indicating meaningful variation among response ratings.

For IF, mean ratings were 4.33 for preserving original content and 4.22 for preserving writing style, with standard deviations of 1.08 and 1.13 respectively. Positive average evaluations therefore coexisted with uneven experiences of preservation.

These figures are descriptive. The 17 and 54 observations represent responses, not independent participants, and the feature comparison was not a randomized experiment. The ratings cannot establish that revision causally improves actionability or safety.

More fundamentally, favorable judgments about the written response did not resolve concerns about whether AI-mediated support represented genuine interpersonal engagement.

The recipient may evaluate the effort behind the message

Peer support involves more than the linguistic properties of a response. A supportive message can also communicate that another person paid attention, considered the recipient’s circumstances, and voluntarily made an effort to respond.

Participants questioned whether AI-generated language would convey that attention in the same way, even when the resulting text appeared empathetic. Some expressed principled objections to AI involvement in emotionally sensitive exchanges.

This creates two different evaluation questions:

• For the writer: Did assistance help express an intended message while preserving control and personal involvement?
• For the recipient: Does the resulting communication convey the human attention and care the recipient expects?

PeerPen provides evidence about the first question through participants’ interactions and accounts. It does not directly establish how actual recipients react to AI-assisted messages.

That limitation matters when translating response-quality ratings into platform policy. An intervention that lowers writing effort for supporters could still be unwelcome to recipients who place value on direct human engagement.

Authorship uncertainty could extend beyond AI-assisted messages

The authors identify a further potential consequence at the community level.

If AI assistance becomes widespread, recipients may no longer know whether a supportive message reflects someone’s personally composed thoughts or a generated response. Participants anticipated that this uncertainty could affect even messages written entirely without AI.

The paper calls this proposed phenomenon authorship spillover.

Its significance is that uncertainty about individual messages could change expectations across the community. A genuinely human-written response may be received with suspicion because similar language could have been produced automatically.

The study does not demonstrate that such deterioration has occurred. Authorship spillover is a participant-anticipated mechanism, developed in the authors’ discussion rather than observed as a platform-wide trust effect.

Nevertheless, it identifies a collective consideration for deployment decisions: individual users can benefit from AI assistance while other community members bear some of the uncertainty about message provenance.

Platform design must account for authorship and community rules

For product teams, the practical implication is to treat authorship control as a distinct design objective alongside writing efficiency.

A graduated interface could begin with prompts, outlines, or suggested phrases, offer localized revision when the user has drafted a message, and reserve complete generation for situations where stronger assistance is deliberately requested. Accepted changes should remain individually inspectable, editable, and reversible.

For community operators, provenance raises additional decisions. Policies could distinguish minor AI-assisted revisions from primarily generated responses, define appropriate disclosure practices, and allow restrictions on sensitive interactions. Recipient preferences also deserve consideration rather than assuming that writer consent is sufficient.

These are Cognaptus operational interpretations of the study’s design recommendations, not validated product interventions.

Privacy requires separate governance. Embedding AI within a platform may provide more control than copying sensitive posts into external tools, but integration alone does not establish appropriate model access, retention, personalization, or permitted data use.

The relevant evaluation therefore extends beyond acceptance rates and text-quality scores. Platforms considering deployment should examine editing burden, perceived authorship, recipients’ responses to disclosed AI involvement, and trust across different community rules.

What the evidence can support

PeerPen offers an exploratory account of how AI assistance changes the experience of writing support. Its qualitative findings are informed by hands-on interactions, interviews, and thematic analysis, but several boundaries constrain deployment conclusions.

The sample was small and self-selected. Eight of the 15 participants primarily identified as community observers, all reported English as their first language, and the tasks occurred in a simulated rather than live community setting. The single-session design also leaves sustained writing improvement and learning transfer untested.

Most importantly, subjective safety ratings are not clinical safety assessments, and participants’ expectations about recipients cannot replace evidence from recipients themselves.

The study therefore supports testing particular interface and governance choices, rather than asserting that those choices already improve trust or supportive outcomes.

The human contribution is a design decision

PeerPen’s contribution is to make the production process relevant to evaluating AI-assisted communication. The same general quality of written support can coexist with different perceptions of who initiated the message, who retained control, and whether the exchange represents meaningful human participation.

For emotionally sensitive platforms, retaining a final approval button addresses only part of that problem. A more informative design question is how the system distributes communicative intent, revision authority, and responsibility between the person and the model.

The next empirical requirement is to test those allocations with both writers and recipients in actual community settings. Until then, selective assistance and transparent governance are defensible design hypotheses, not guarantees of authenticity.

Cognaptus: Automate the Present, Incubate the Future.