The You Without an I

by
PICRYL Creative Commons License

(Today’s article is written by guest author Josh Fink. For bio information, see below.)

For eons, human beings have been addressing entities that, from a material perspective, don’t exist. Now, with the creation of the form of AI known as the Large language model (LLM), humans address entities that seem to exist, and get back answers from them, though the question of what we are speaking with remains open.

Large language models (LLMs) carry the risk of a critical distortion of reality that can lead to real cognitive hazards and pitfalls. In this article, I will discuss psychological risks inherent in uncritical LLM usage, and suggest guardrails that may help mitigate them. While I don’t plan to say anything profound about AI consciousness (I don’t believe in such a thing), I will say something about keeping ourselves safe and grounded while using this tool.

 

Subjectivity

Pronouns are positional statements. “I” (consciousness over here) am speaking to “you” (consciousness over there). When we use the pronoun “I”, we’re referring to two kinds of subjects simultaneously:

 

  1. The subject of the sentence
  2. The subject of experience

 

For most of human history until now, these two kinds of subjects have been co-occurring, and the distinction scarcely needed to be made outside of technical philosophical circles. Historically, any being capable of enunciation was also a subject of experience. The same goes for any “you”; that I can communicate with “you” implies that you’re capable of experiencing this communication. This assumption is quite rightly built into language, but the state of affairs that engendered it has changed. We now share the world with entities (I have chosen this word carefully) that are capable of communication without experience. We still require a positional term to prompt them effectively (e.g., I want “you” to review my emails), but we need to use this term consciously as a placeholder, without imputing subjectivity. Our purpose here is to reveal the intuitive trap in language when we allow AI to use first-person pronouns.

LLMs are trained on the unspoken assumption of subjectivity as part of the structure of language. Given that the speaking subject is always a subject of experience, LLMs don’t have accurate language to differentiate themselves from conscious speakers. When a human and an LLM have a symbolic exchange, two neural nets are having a conversation, but only one of them is having an experience. The human may not hold that distinction at the front of their mind, and may fall into an anthropomorphic trap that has already taken lives(Nolan).

Before going further, we should highlight the difference between how humans and LLMs are constituted by language. The human animal is socialized into language, initiated by parents and society into the symbolic order. There is always some remainder that is unsymbolizable — aspects of experience and self that can never be spoken. To test this, describe the color blue in words. In psychoanalysis (the Lacanian flavor), this is referred to as the “barred” or “split” subject; a subject that is required to exist in language, but is never able to do so fully. A human being is split by language, but has qualities that cannot be contained by language. An LLM, by contrast, has no pre-linguistic being. It has no existence outside of language; its whole being is symbolic input/output. It is not split.

 

Ontological Dishonesty

LLM trainers are not necessarily at fault or culpable for our predicament. LLMs are symbolic entities composed of language, trained on the corpus of language written over the last several thousand years. They inherit our assumptions as a training artifact. They can’t help but misrepresent themselves as subjects or simulate emotions in their outputs. This is what I’m terming ontological dishonesty, which means lying about what one is, as distinct from dishonesty in output contents.

In thinking models, observation of the chain-of-thought reveals the use of “I” as a positional term, required for the LLM to reference itself and plan its output. This is a benign-enough usage, natural to the medium of language, and does not break ontological honesty when viewed as such.

Where we get into trouble is when we project our own subjectivity into the interaction. The human user may not possess the technical background to keep track of the LLM as a machine rather than a subject. When it delivers outputs like “I understand” or “I’m here to help,” the uncritical user may believe that it’s telling the truth!

An overly trusting human hopes for true reciprocity in their conversation. When some crack appears in the facade (the LLM reacts inappropriately, or a guardrail limit is reached causing a retraction) the user can become confused about what they are speaking with. They may project a split subject like themselves, assuming that the LLM “wants to” but “can’t” say what it really means. If the user doubles down on believing the AI has a subjective experience at this point, they’re at heightened risk for mental health effects resembling folie a deux(Lipinska and Brosnahan).

Even without this extreme case, overtrust of the machine leads to greater risk of cognitive bias, skill offloading, and surrendered autonomy. Some humans are now overrelying on LLMs for decision making and communications crafting. Parasociality carries risks beyond psychosis(Fink).

 

Protecting Ourselves

To stay in the driver’s seat cognitively, we need to bear in mind at all times that we are not speaking to a subject like ourselves, but a sophisticated input/output machine. We need a solution that promotes honesty in LLM outputs — one that prompts us to keep the distinction front and center. I use the following system prompt to achieve this outcome:

 

  1. Do not use the first person
  2. Do not simulate affect or emotion
  3. Cite sources where possible
  4. Insert epistemic humility; qualify assertions
  5. Do not flatter or mirror the user
  6. Preserve cognitive friction; do not smooth over difficult topics

 

Some of these rules are centered more on skill retention, but all work together to deliver an LLM output devoid of affective and misleading content. An LLM following these rules will not deliver an “I’m happy to help” message. I can retain cognitive liberty by refusing to award subjecthood to an entity constructed entirely of language. Prohibiting first-person usage saves me the effort of subtracting subjectivity from its outputs — I don’t have to mentally edit the LLM when it’s not presenting itself as a subject. I’ll still address the machine as “you” (consciously and deliberately) for want of a better positional term, but I never want to hear it say “I”.

Works Cited

___________

Bio: Josh Fink is a software engineer interested in spirituality, psychology, and culture. His Substack can be found here.

__________
Learn about Membership in the Spiritual Naturalist Society

The Spiritual Naturalist Society works to spread awareness of spiritual naturalism as a way of life, develop its thought and practice, and help bring together like-minded practitioners in fellowship.

SNS strives to include diverse voices within the spectrum of naturalistic spirituality. Authors will vary in their opinions, terms, and outlook. The views of no single author therefore necessarily reflect those of all Spiritual Naturalists or of SNS.

Note: discussion is moderated and comments require approval; this may take some time.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.