What do the large language models behind human-like conversational AI really know and what does it mean to live alongside them?
A new book Understanding Conversational AI by Thierry Poibeau offers a critical and interdisciplinary exploration of large language models (LLMs), examining how they reshape our understanding of language, cognition and society. Drawing on philosophy of language, linguistics, cognitive science and AI ethics, it investigates how LLMs generate meaning, simulate reasoning and perform tasks that once seemed uniquely human from translation to moral judgement and literary creation.
The book explores the limitations of these models, their embedded biases and their role in processes of automation, misinformation and platform enclosure while reflecting on how they prompt us to revisit fundamental questions: What is understanding? What is creativity? How do we ascribe agency or trust in a world of synthetic language?
Recent findings are suggesting that non-expert poetry readers prefer AI works to poems written by humans. Thierry Poibeau reminds us how “aesthetic judgment in poetry often involves elements that are not easily codifiable, such as originality, emotional resonance, metaphorical depth, and cultural embeddedness”. Poibeau sheds light on why AI-generated poems can be appealing, by highlighting how literary value is “shaped by evolving norms within particular reading communities, making judgments of poetic merit historically contingent and socially negotiated”.
AI Poetry in motion
Excerpts from Understanding Conversational AI
“The arrival of AI-generated poetry raises fundamental questions about whether current evaluation criteria, often rooted in human experience, intentional expression, and historical context are adequate for assessing texts produced through large-scale statistical recombination.
As large language models generate increasingly fluent and plausible verse, their outputs may challenge existing notions of aesthetic authenticity, precisely because they blur the boundary between craft, imitation, and genuine creative insight.
These systems are capable of reproducing recognizable poetic forms, stylistic conventions, and affective tones, raising the question of how such texts should be evaluated when traditional assumptions about authorship and intention no longer clearly apply.
However, despite this formal competence, recent studies reveal recurrent stylistic tendencies characteristic of large language model–generated poetry. GPT-4, for instance, shows a strong preference for quatrain structures, a frequent reliance on iambic meter and end rhyme, and a tendency to employ recurring lexical choices such as heart, whisper, or dream.
These outputs often reflect a flattening of emotional and metaphorical complexity, favoring literal formulations and conventional poetic tropes over ambiguity, innovation, or semantic depth.
In comparison to human poetry, LLM-generated verse appears more homogeneous and less nuanced, and is less capable of producing the kinds of conceptual tension or unexpected imagery often found in more original human compositions.
In addition to these stylistic patterns, recent reader evaluation studies suggest that some AI-generated poems are judged comparably to human-authored texts when assessed blind.
However, once authorship is disclosed, evaluations tend to decline, indicating a persistent skepticism toward machine-generated poetry. This reception dynamic highlights the ongoing tension between the formal fluency of AI-generated verse and readers’ perceptions of authenticity and creative agency.
Interestingly, readers often find AI-generated poems easier to interpret. They can more readily grasp the images, themes, and emotions, which are typically presented in a more accessible and transparent manner than in the often denser and more ambiguous work of human poets.
As a result, readers may develop a preference for these texts and mistakenly interpret their own ease of understanding as evidence of human authorship. When the machine origin of the poem is revealed, this interpretation shifts, and the same textual features that previously facilitated comprehension may be reinterpreted as signs of superficiality or lack of depth.
Such findings should be interpreted with caution. They do not demonstrate that AI-generated poetry is superior in literary quality, but rather that it tends to conform to familiar forms and accessible conventions. In contrast, human-authored poetry frequently relies on layers of complexity, allusion, and ambiguity that demand substantial background knowledge in literature, history, and poetic traditions. Interpreting such works is a cognitively demanding task, and it is therefore problematic to ask lay readers to evaluate poetry in a meaningful way without accounting for differences in expertise and interpretive competence.
Beyond questions of evaluation, the reception of AI-generated poetry raises broader issues of attribution, disclosure, and creative ownership. Literary institutions, publishers, and competition organizers increasingly require authors to disclose the use of AI tools, reflecting uncertainty about how machine-generated texts should be situated within existing frameworks of creativity and value.
The discomfort that emerges when readers discover that a poem was generated by a machine is not merely a reaction to deception but an indication that aesthetic judgment remains deeply entangled with assumptions about intention, experience, and authorship.”
Thierry Poibeau is Director of Research at the Centre national de la recherche scientifique (CNRS), Paris and head of the LaTTiCE laboratory in Paris, France.
He is a member of PRAIRIE-PSAI (Paris AI Research Institute – Paris School of Artificial Intelligence) and is also an affiliated lecturer at the Department of Theoretical and Applied Linguistics (DTAL) of the University of Cambridge. He works on natural language processing (NLP), particularly focusing on information extraction, question answering, semantic zoning, knowledge acquisition from text, and named entity tagging.
A weekly e-mail in English featuring expertise from scholars and researchers. It provides an introduction to the diversity of research coming out of the continent and considers some of the key issues facing European countries. Get the newsletter!
Source:
theconversation.com
