This floated past my feed on LinkedIn, and should have been captioned et tu, Brute?

The joke being, of course, that it’s from Cards on the Table, by Agatha Christie, published in 1936…

Christie isn’t the only victim. Can you recognise this paragraph?

It’s the opening of Chapter 5 of Mary Shelly’s Frankenstein. Published 1818. Kinda ironic, when you think of it.

In this post I’d like to cover a bit of AI in writing – from the jokes, to how to defend yourself against accusations, to some of the development in this unfortunate field.

The Obvious

AI for curing cancer is good^*, AI for anything creative is bad. In the same sense that a nuclear disaster fuelled by garbage bin juice is bad.

^* Nothing wrong in developing AI for cancer, but often claims are overly-exaggerated. This is because there’s a hype bubble, and “AI” is actually a marketing term that purposefully conflates multiple technologies and use-cases. There’s nothing wrong with chainsaws either, but don’t use them for surgery or call any gardening tool a chainsaw. The industry does this on purpose, which frustrates those of us who understand the nuances.

Anyway. Eric and I (my co-conspirator at Purple Toga Publications) have a zero-tolerance for AI in creative uses. That applies to writing, obviously, but also to any related art. We made it clear to anyone who participates in our anthologies that it won’t be tolerated even for joke marketing assets. It just has no place in a space that’s dedicated to human creativity.

Unfortunately, our day jobs, the one we’re sadly forced to keep in order to eat and survive inclement weather, involve AI. So we do get exposure to the technology and can at least speak about it without falling into either side of the hype. This post covers a bit of my experiences, and may address some of the things you might have heard and are curious about.

Side note: We’ve been building custom AI and following up on emerging tech for years, as tools of the trade. If you think about AI as a tool and make an obvious analogy, the whole situation becomes surreal. Replace “AI” with “Angle Grinder”. You want to buy a house, but the whole discussion with your architect is about which angle grinder the builders are using. Boeing & Airbus are advertising the angle grinders in airplane construction. CEOs advertise that surgeons would soon be replaces by angle grinders. Your boss, who hasn’t touched tools for a decade, insists that since we bought this expensive bit, we need to use it more so start hammering nails with it. And we’re just sitting there, knowing when you can, can’t, and shouldn’t use the tool, and have to nod along.

Other-side note: for the rest of this article, I’ll use “AI” as a short-hand for “LLMs and NLP”, the machine learning technologies that are concerned with language and are in the cross-section of the hype and writing. I’m letting the marketing get to me…

The Accusations

There’s a lot going on about authors being accused of using AI, editors using AI, and books being pulled from publications deals. We’re in the immensely lucky position that we are entirely unaffected by 6- and 7-figure cash publishing deals (no one said it was good luck 😅), but that must still be very frustrating for the likes of Ms Christie, Ms Shelly, and other innocent mortals.

A key problem is that AI-detectors are very fallible (mostly being AI themselves). Since LLMs were training on human writing, it’s no wonder that they come close, y’know, human writing, and it isn’t always easy to tell. People bandy about AI accusations easily, and in a way that’s similar to Brandolini’s law, the amount of energy used to disprove such claims often far exceeds the effort it took to make the accusation.

There are articles doing the rounds on how to protect yourself from accusations. These cover anyone who might be accused, from students to authors. Mostly they come down to keeping evidence – word count logs, document histories, research logs (aks browser history) etc. Luckily for authors, spending an evening of “writing” involving alternately staring out the window and at an errant comma and resulting in a net -20 word count is considered normal 😂

Many writing programs (from Scrivener and yWriter to Google docs) do that automatically, there are word-count services, and we all keep a jumble of notes and browser history in some form or another. Since books (outside of trashy romance) normally take a lot longer to produce than college essays, this shouldn’t be a problem if you’re just conscious enough to keep the notes around.

Anthropic’s Watermarks

Another thing you may have seen is about how Anthropic, the maker of Claude AI, now have it ‘watermark’ the writing in a way that travels with the text and is resistant to light editing. Technically, because generative language models are probabilistic, they do it with a special key that gives a small ‘nudge’ to the next word selected from the set of likely next words. Given enough nudges, you could in theory verify that it’s statistically improbable that the text wasn’t nudged, and is thus highly likely AI generated.

This mechanism is based on Google’s SynthID-text algorithms, previously published in Nature. It’s driven by EU transparency laws, but applied globally. Whether other model providers pick this up remains to be seen, especially due to the initial user backlash against Anthropic.

The problems with this approach are:

• Quite often the set of probable next word is very small, so a nudge wouldn’t register much difference.
• It works on long text only, where you can accumulate enough such nudges to be statistically significant.
• You need the key (a mathematical function) to be able to verify this. Anthropic has announced an API is coming, so people could check. It will obviously work only against their models (Claude), not generically.
• As they themselves note, it will show if Claude was heavily involved in direct generation, not post-editing (like spell-checking). So it’s more about how much of the text was generated by Claude.
• Light editing won’t affect it (because it will still detect a large-enough number of nudges within the text), but other methods will (from heavy editing, to using a different model to edit, to translating back and forth to a different language).

So it’s an interest approach (mathematically speaking), but as it’s far from infallible it won’t really affect the landscape of AI usage and accusations.

Recognising AI

Can you recognise AI? I saved this for last for a reason.

Current AI detection is based on trying to recognise AI catchphrases and stylistic choices. It’s not when a work is “doing the work” or is “load bearing” in a sentence — it’s genuinely a matter of voice. We could delve deeper. But I’ll let you sit with that for a moment. And honestly? I’ll say it’s an ephemeral property — and you’d be right to push back on that.

Sorry, couldn’t resist 😂

So yea, there are definitely verbal tics that are very common to LLMs, usually more so than simply counting emdashes. But those same phrases come up in everyday speech and writing, as do emdashes. It’s more a matter of degree: LLMs were trained of writing in general, and then reinforced on “this is good writing” examples, so those catchphrases, constructions like “Not X – Y”, and emdashes just appear in much higher concentration than in general writing.

For example, Mary Shelly’s Frankenstein has about 1.4 emdashes per 1,000 words, while Jane Austen’s Pride and Prejudice has about 3.1 per thousand. My own books have about 5.8 / 1,000 (go on, throw those emdash memes at me, I deserve them). In contrast, when I tested AI writing it comes to about 12-13 per 1,000.

So could we draw conclusions and test the frequency? Not so fast. The aforementioned Cards on the Table by Christie clocks a whooping 15 emdashes per 1,000 words! LLMs must really rate Christie as a top writer (as do we, to be fair).

Are we lost, then?

Well, no. As I said, Eric has been subjecting himself to reading AI short stories, and I unfortunately have to deal with much AI writing at work. We do it for a reason, and that is that just like any art you acquire taste and discernment by experiencing good as well as bad examples. And there are some fundamental properties of AI that it just can’t hide, which are far more pervasive than some secret mathematical function.

Firstly, AI has a particular voice. Just like a person that you learn to recognise their style, so do most AI when left to the ‘default’. Even when instructed to imitate other humans, this still comes across like someone in a bad disguise 🥸 It does change somewhat over time and between vendor models, but it’s that feeling where you know in your gut. I’ve done it too with visual art: learning from real artists what are the signs they look for, working through identifying them myself, and now I can tell at a glance and be pretty accurate (at least according to the periodical online ‘human or AI’ tests I take).

Secondly, having no embodied experience, being a statistical model of language rather than a consciousness navigating a physical world, it messes up space and time repeatedly. Everything happens on a Tuesday (a trope that dates back to the 90’s, which AI regurgitates at alarming rates), and yet the passage of time is wobblier than in a bad Doc Who episode. Things move around space (with or without actual travel) in nonsensical ways. Not that human authors don’t make mistakes like this on occasion, but the sheer amount of such errors in even a simple short story is staggering. I can’t even imagine how it might handle a large scale novel.

Lastly, AI writing is vacuous. It spins words trying for smart similes and turns of phrases, and when you read them all you can think is “huh? that makes no sense.” There’s the budding writer trying to be clever, and then there’s AI. You can feel the lack of human empathy in it.

All of these can be human traits, but when coming together at the scale that AI seems to pour them into the words it churns out (I won’t call it writing), it’s one of those “once you see it you can’t unsee it” – it becomes blatantly obvious when AI writes something from a story to a blog post to a social media post. Just… yuck.

This is backed by research, to a degree. A study called StoryScope asked models to generate multiple stories and then had them analysed, and found that they all have certain features that tend to cluster together. Human stories, in contrast, occupy a different area and are much more diverse in range.

This research is limited to short (~5,000 words) fiction stories written to a prompt, but it does confirm that it’s not so much any particular aspect (phrase or emdash) as an inherent ‘fingerprint’ of AI voice.

The even better way

And, of course, there are meta-ways of gaining confidence that a piece of fiction is AI written (rather than AI-edited, or just poorly written). That conventional wisdom in publishing circles is to ask the author incisive questions about the choices the made for character motivation and world building. Someone who relied on AI won’t be able to answer; it’s almost like they read it for the first time with you, trying to come to grips with the story. A human author, on the other hand, wouldn’t shut up about that stranger on the train that reminded them of the dream they had and how it all made it into the story.

The best tool – whether in fiction, or academic writing, or anything really – is to test for pride and ownership. Art (and academic papers) are born of struggle, and it’s visible as soon as you scratch the surface.

Lastly…

On the assumption that you’re writing (and reading) like a person, don’t stress too much. Read what you enjoy, write what catches your fancy, have a bit of fun in this world on the brink of dystopia.

If you want to build an innate AI-detector, you’ll need to expose yourself to some trashy works until you learn to recognise it within a couple of sentences. It does have the downside of completely burning certain (valid) expressions to the point you’ll want to puke when you read them, even when it’s someone whom you know didn’t use AI. It’s a good skill to deal with misinformation, these days, but your mileage may vary.

If you’re writing and are worried about being accused of using AI, all you need to do it as listed above – keep your records, keep writing, and keep reading human-written works. Not that anyone ever blames me, but I can happily point at my very much pre-AI novels, point at how they were used to train the AI (proven in court, more or less), so of course it can sound like me – it’s trying to imitate good writing! Idiots can sit this one out.

Anyway, as a finale to leave you with some good taste, for those who’d prefer a more modern cartoon that poke fun at the industry, I can heartily recommend Tom Fishburne:

(This is literally what Grammerly turned into – shoving both AI-writing and AI-detection into what used to be a glorified spell-checker).

---

Why don’t you try reading my very much non-AI free short stories and novels? Not only are they entirely human written, they also don’t include any AI, just your usual ghosts and bad politicians.

Bonus!

You know who’s using AI? Spammers! I get so many emails eager to tell me about my own books 🤦 Like, seriously, I wrote them, I know what they’re about. I also wrote the blurb you’re quoting back at me, pretending it’s an insight. None of that, however, reaches the following level of dumbfuckery: