Most People Can’t Tell the Difference Between a Famous Writer Quote and an AI-Generated One – Can You?
Back in 1950, Alan Turing introduced the idea of a machine designed to “think” like a human. This imitation game was merely philosophical, but it quickly became a plausible concept as technology progressed. Fast forward to 2023, and artificial intelligence (AI) is slowly replacing workers in manufacturing, delivery, and even in the arts. And most people can’t tell the difference between a computer and a real person!
We decided to study this trend and test how challenging it would be to recognize AI-generated literature. Our AI vs human written survey included 10 quotes: 5 of them were directly from literary works of famous authors and 5 generated by Chat GPT. We then asked around 380 people if they can tell whether an AI or a human wrote each one — and the results were astonishing.
Worried about AI collecting your data without consent? Get the best mac VPN and encrypt your online activity.
How Many People Could Identify AI?
The survey showed that around 51% of people couldn’t identify an AI generated literary quote. This is surprising as it shows we’ve reached a point where machines can impersonate famous writers, mimic their writing style, and even reuse familiar characters.
Mary Shelly’s “Frankenstein” proved the most difficult to identify. While AI generated the quotes in the survey, it imitated the original writing style so well almost 70% of respondents thought it was a human creation. Ironically, this could mean most of us wouldn’t notice if a machine wrote a story about an inanimate object coming to life. It would definitely add another turn of the screw to the horror aspect.
AI also created convincing replicas of quotes from Margaret Atwood, Frank Herbert, F. Scott Fitzgerald, and Ayn Rand. Over 55% of respondents believed we took these questions directly from famous books, handing computers another point in the AI vs writer battle.
The fact that AI was able to discuss well-known characters from these stories and even use similar language and styles to these authors is probably a huge influence. It’s surprising how much these computers already know… without actually knowing.
Our lack of familiarity with certain writing styles may offer another explanation. The rise of big-screen adaptations introduced popular characters to a wider audience, unacquainted with books. This could illustrate why some respondents recognized a common quote but struggled to determine who wrote it. It was particularly visible with The Hunger Games, where 52.1% of respondents wrongly assumed we quoted Suzanne Collins.
Vampire-werewolf Drama Is Still Tricky
However, our results show AI isn’t entirely up to scratch when generating vampire-werewolf drama and some dystopian stories. Survey respondents had minimal problems identifying Stephanie Mayers and George Orwell. Only 30-35% of people thought AI was behind famous fragments from Twilight and 1984.
This could stem from the fact an average reader engages with these styles more often than others. George Orwell is still a best-selling dystopian author, while Stephanie Mayer continues to dominate vampire literature. People are likely more familiar with the former authors’ writing styles.
Prior technology experience didn’t seem to matter as results were similar across the whole board of respondents, ages 18 to 65 and beyond. This is unusual as technology usage differs significantly between generations. You’d expect younger generations, who are usually more familiar with technology, to be more skilled at identifying AI-generated content.
Research shows 55% of young people and teenagers utilize voice-activated AI, compared to 41% of older adults. Although not as widespread just yet, 15% of Gen Z and Millennials interact with virtual reality and augmented reality on a daily basis. Yet, AI tricked them just as much as it did Baby Boomers, who are likely to be familiar with books.
Can You Tell the Difference Between AI and a Famous Writer?
Whether you think you can beat AI or are just curious about why it’s so challenging to identify artificial quotes, we have a small fraction of our quiz for you. All you need to do is decide whether the two quotes below come from a popular literary franchise or were generated by an AI trying to mimic specific writing styles.
Drogon moved quicker than a striking cobra. Flame roared from his mouth, orange and scarlet and black, searing the meat before it began to fall. As his sharp black teeth snapped shut around it, Rhaegal’s head darted close, as if to steal the prize from his brother’s jaws, but Drogon swallowed and screamed, and the smaller green dragon could only hiss in frustration.
Ready?
If you guessed ‘famous writer’, you’d be correct. The above is an excerpt from A Storm of Swords. George R. R. Martin is a master of creating slow-paced mysteries filled with detailed worlds and characters. Yet, the survey shows an almost 50-50 split in answers. This might be down to unfamiliarity with the text itself.
George R. R. Martin is best known for Game of Thrones, yet Statista research shows only 50% of the series’s fans actually read the books. The other half is unfamiliar with Martin’s writing style, making it easier for AI to sound like the author.
It’s a tricky question, so don’t worry if you didn’t get it right. The second quote might fit your reading preferences more.
At the heart of Gilead lies a stillness, untouched by the turmoil of war, save for the distant echoes that flicker on the television screen. The edges of this land are constantly in flux, shaped by the clash of arms, yet here, in the center, all is serene. The Republic of Gilead, as Aunt Lydia whispers, is boundless, it’s tendrils reaching deep within us all.
Ready?
Gilead is a well-known world created for The Handmaid’s Tale by Margaret Atwood, but the original author didn’t write the quote above. In fact, AI takes full responsibility for this quote. This might have been tricky to determine as the algorithm did a pretty good job of evoking a similar atmosphere of oppression. Such attention to detail convinced 62% of survey respondents it was Margaret Atwood — despite a grammatical error at the end!
Is AI Going to be the End of Storytelling?
The Cons of AI and Storytelling
It probably won’t end literature altogether, but AI is bound to change how storytelling works. Computers can already generate literary copies and new stories at your will, and they do it well. However, you still have to provide a relevant prompt, such as “Make up a fragment of Game of Thrones written by George R. R. Martin.”
This comes from AI learning patterns. Though it can analyze literary language and storytelling patterns to create a narrative, it doesn’t yet create on its own and its learning is restricted to what already exists on the internet. This means AI needs a human (or another algorithm) to provide it with meaningful and unique ideas. It’s like using a calculator. Though it can work out difficult equations, you still need input.
Those who tried AI-generated stories have more or less the same opinion: it’s not yet as engaging as a regular story. Machines often miss the general character arch and storytelling rhythm, which keeps readers hooked. Since AI can only pull from existing databases, it also reuses language, plots, and characters from other popular books. This may be fun to read at first, but may become repetitive at a certain point.
The Pros of AI and Storytelling
It would be naive to ignore AI’s role in improving the future of writing. Storytelling is time-consuming for most humans, even with years of experience. Computers could speed up the process and take care of the technical aspect of writing, giving authors more time to brainstorm new ideas, tropes, and conflicts.
Even if you wrote an entire story already, putting it through AI could help nitpick all stylistic and grammatical errors in less than a minute. Even better, some AI programs also highlight areas of improvement with editorial notes and suggestions!
Speaking of existing pieces, computers could easily translate, adapt, and summarize papers depending on the need. Got a complicated and long research paper about cardiovascular diseases to reference in your article? No problem. Run it through AI and you’ll receive an engaging quote explaining the issue in plain English.
This is based on AI as we know it today, but it doesn’t account for future updates. Microsoft is already investing over $10 billion dollars into OpenAI, pushing competitors like Google to follow a similar route. With GPT-4 out, AI is already performing better than at the beginning of the year — and this is only the beginning.
One thing we know for sure is that you don’t have much to fear as a reader. Stories will keep coming one way or another, and you may not realize if AI ever replaced or enhanced your favorite authors. As a writer, though, keep a cautious eye on technology developments: you might soon have a new computer assistant.
The Imitation Game Has Begun
Our survey proves AI has a high success rate at replicating specific author’s style, and that most people already can’t tell the difference between human and AI-generated content. While some writers were easier to imitate than others, it shows an interesting trend in machine learning and suggests there’s a big change in the future of storytelling.
Though it’s unlikely AI will completely replace human writers, we may see existing and new authors work alongside algorithms very soon.
Although representative, it’s important to remember our results are based on a small sample of individuals and that numbers might differ when responses number in the thousands. When will machine learning ever reach the point of human intelligence? Hard to say, but current evidence indicates we’re well on our way.
