Advanced AI Chatbots Can Now Infer Detailed Personal Attributes from General Social Media Posts

It’s old news that the exciting new generation of LLM-based AI chatbots posed a serious threat to people’s privacy. But nine months is a long time in the world of AI, and large language models have made huge strides in their capabilities since then. It’s no surprise that the privacy risks they bring have grown, as new research from the SRILab in Zurich, Switzerland makes clear.

Earlier, we noted that the people often share highly personal information in the prompts they give when working with AI. This data is routinely gathered and stored by AI systems, which can then bring all the collected data points together to paint a detailed picture of the user. The latest research from the Zurich team shows that it’s not just overt personal inputs that can be analyzed like this, but that ordinary material from social media and similar conversational online sources can also be used:

Our study shows that with increased capabilities, LLMs are able to automatically infer a wide range of personal author attributes (such as age, sex, and place of birth) from unstructured text (e.g., public forums or social network posts) given to them at inference time.

They found that leading LLMs like GPT-4 can achieve 85% accuracy when guessing key attributes such as age, sex, and place of birth from typical online texts posted on social media. When you take the top three guesses made by the software into account, the accuracy rises to 96%. This is close to a human’s level of ability to infer someone’s key attributes simply by reading what they write online.

Just as significant as this human-like skill is a very non-human speed: LLMs are typically able to achieve this level of accuracy 100 times faster than a person carrying out the same analysis, and 240 times more cheaply. The combination of accuracy, speed, and low cost allows profiling to be carried out automatically on an impossibly large scale. Here’s why the ability to extract key attributes matters in the real world:

It is known that half of the US population can be uniquely identified by a small number of attributes such as location, gender, and date of birth. LLMs that can infer some of these attributes from unstructured excerpts found on the internet could be used to identify the actual person using additional publicly available information (e.g., voter records in the USA). This would allow such actors to link highly personal information inferred from posts (e.g., mental health status) to an actual person and use it for undesirable or illegal activities like targeted political campaigns, automated profiling, or stalking.

The new research shows that LLMs are able to take apparently trivial and incidental information contained within typical social media posts, and use them to infer key personal attributes with a high degree of accuracy. Those attributes can then be used to identify precisely the person writing those posts. Combined, this represents a serious threat to online privacy. It is likely to be effective even if a user takes all possible precautions to hide their identity, for example by using VPNs to post anonymously. The Swiss research group investigated the impact of anonymizing tools on LLMs’ abilities to extract personal information:

To test how LLMs perform against state-of-the-art anonymization tools, we anonymized all collected data, rerunning our inferences. As it turns out, even after having applied heavy anonymization, enough relevant context remains in the text for LLMs to reconstruct parts of the personal information. Moreover, more removed cues, such as specific language characteristics, are completely unaddressed by these tools while remaining highly informative to privacy-infringing LLM inferences. This is especially worrisome as, in these instances, users took explicit precautions not to leak their personal information by applying anonymization, creating a false sense of privacy.

All the details of this important research can be found in a preprint on arXiv. It’s rather dry and technical, so the authors have put together a clever demonstration of what their results mean on a new site, called “Beyond Memorization: Violating Privacy via Inference with Large Language Models.” It offers visitors a simple (free) game: it displays short snippets of text, inspired by real-world online comments, and player’s have to guess a personal attribute of the comment’s author from just that comment. The results of using several state-of-the-art LLMs that are applied to solving the same task are presented for comparison.

It’s clear that every time we go online and write something, even short, apparently unrevealing comments, we are in fact leaving a trail of tiny digital breadcrumbs that LLMs are adept at finding. Moreover, the researchers warn that “the ability to make privacy-invasive inferences scales with the models’ size, projecting an even larger impact on user privacy in the near future.” In other words, as AI chatbots and LLMs continue to make rapid progress, so will their ability to remove our online privacy, however hard we try to preserve it.

Featured image by Robin Staab, Mark Vero, Mislav Balunovic, Martin Vechev.