ChatGPT Is One Year Old: Here’s AI’s Next Attack on Privacy, and What to Do About It

Although it’s hard to believe, it’s only a year since ChatGPT appeared and ignited the current frenzy over AI and large language models. As an article in the New York Times makes clear, the feverish pace of development since then has been driven by companies that are afraid of being left behind in the race to exploit the Next Big Thing. Products and services were released quickly, often before they were fully finished, and with no real understanding of what their impact would be.

As we have reported, it turns out that the new wave of AI technology has serious negative consequences for privacy. These include:

• The leak of personal information from the prompts used with systems like ChatGPT
• The naturalistic nature of the interactions leading to people dropping their guard and revealing intimate details of their lives
• The use of AI to harvest personal data from online services
• The use of advanced AI chatbots to infer detailed personal attributes from general social media posts

The constant, rapid flow of technical advances in this field makes it hard to stand back and see the bigger picture. One leading commentator who has managed is Bruce Schneier. He’s a respected security expect who often writes about privacy issues in his books and on his blog, Schneier on Security. He has just republished a thought-provoking essay about AI, which previously appeared on the Harvard Kennedy School Belfer Center’s website.

Schneier begins by echoing dozens of posts on this blog, noting that the internet has enabled mass surveillance by tracking everywhere we go online and everything we do there. As he wrote in another blog post a few weeks ago, “Surveillance is the business model of the internet – Al Gore recently called it a “stalker economy.” That’s now widely accepted, but Schneier points out that AI’s new capabilities make the situation even worse, by moving from mass surveillance to mass spying.

The key capability of AI that allows it to carry out intelligent spying, rather than simple surveillance, is summarization. As ChatGPT and similar services have demonstrated, they are already quite good at summarizing documents, describing images, even transcribing and summarizing videos. They can also use their summarizing capabilities to categorize, organize and search through mixed data on a hitherto impossible scale, and present the results as easily digested reports. For example, you could present an LLM with a huge dataset and instruct it:

Tell me who has talked about a particular topic in the past month, and how discussions about that topic have evolved. Person A did something; check if someone told them to do it. Find everyone who is plotting a crime, or spreading a rumor, or planning to attend a political protest.

There’s so much more. To uncover an organizational structure, look for someone who gives similar instructions to a group of people, then all the people they have relayed those instructions to. To find people’s confidants, look at whom they tell secrets to. You can track friendships and alliances as they form and break, in minute detail. In short, you can know everything about what everybody is talking about.

Schneier points out that AI spying won’t be limited to online conversations. The presence of cameras and microphones everywhere will provide even more raw data. Until now, it has been a slow process analyzing that data. Soon AI will be able to pull in huge numbers of multimedia files, and analyze them on a scale and at a speed that are far beyond what is possible today.

With that new capability comes new ways of undermining privacy. Governments will use this kind of spying to facilitate social control. They already use mass surveillance, as Edward Snowden revealed over ten years ago. Mass spying will be even more attractive to them, and even more dangerous for us. Just as companies have adopted surveillance advertising, so too they will readily embrace the new possibilities of AI spying:

Corporations will spy on people. Mass surveillance ushered in the era of personalized advertisements; mass spying will supercharge that industry. Information about what people are talking about, their moods, their secrets—it’s all catnip for marketers looking for an edge. The tech monopolies that are currently keeping us all under constant surveillance won’t be able to resist collecting and using all of that data.

It’s a bleak vision. Schneier concludes: “We could prohibit mass spying. We could pass strong data-privacy rules. But we haven’t done anything to limit mass surveillance. Why would spying be any different?” To be fair, the EU’s GDPR has at least tried to tackle mass surveillance, as numerous posts on this blog have described. Some countries are already trying to use the GDPR to tackle the new privacy concerns raised by AI. The EU is also finalizing an AI Act that aims to regulate the new technology directly. But another article in the New York Times suggests that the pace of change is simply too fast for laws to keep up with AI. Schneier himself has an alternative solution, which he explains in a long and interesting post about AI and trust. He says what we need are “public AI models,” systems built by academia, non-profit groups, or the government, that can be owned and run by individuals:

A public model is a model built by the public for the public. It requires political accountability, not just market accountability. This means openness and transparency paired with a responsiveness to public demands. It should also be available for anyone to build on top of. This means universal access. And a foundation for a free market in AI innovations. This would be a counter-balance to corporate-owned AI.

It’s certainly an idea worth exploring. After all, the stakes are incredibly high: if today’s mass surveillance turns into tomorrow’s AI-based mass spying, privacy will become an even more endangered right.

Featured image by OpenAI.