{"id":26324,"date":"2024-02-26T02:59:41","date_gmt":"2024-02-26T10:59:41","guid":{"rendered":"https:\/\/www.privateinternetaccess.com\/blog\/?p=26324"},"modified":"2024-02-26T02:59:44","modified_gmt":"2024-02-26T10:59:44","slug":"chatbots-wont-win-your-heart","status":"publish","type":"post","link":"https:\/\/www.privateinternetaccess.com\/blog\/chatbots-wont-win-your-heart\/","title":{"rendered":"AI Chatbots Won\u2019t Win Your Heart, but They Will Harvest Your Data"},"content":{"rendered":"\n

A year ago, we wrote that ChatGPT is a privacy disaster waiting to happen<\/a>. One of the issues we mentioned was that the new generation of highly convincing chatbots encourage people to enter personal data without much thought about what happens to it or how it can be misused. When the technology was new, it was a theoretical concern, but a year is a long time in the AI world, and a lot has happened in the area of chatbots.\u00a0<\/p>\n\n\n\n

An article in the Japan Times reports on young women in China routinely turning to sympathetic AI \u201cboyfriends<\/a>\u201d for companionship and comfort.\u00a0 It\u2019s easy to imagine people around the world becoming very attached to these sympathetic chatbots, and less guarded with the personal information they reveal to them.<\/p>\n\n\n\n

When OpenAI launched its GPT (Generative Pre-trained Transformer) store at the beginning of 2024, AI \u201cgirlfriend\u201d apps immediately flooded the new store, even though they contravene OpenAI\u2019s usage policy, which bans GPTs \u201cdedicated to fostering romantic companionship.\u201d<\/p>\n\n\n\n

One of the biggest concerns about these \u201cromantic\u201d chatbots is privacy. If chatbots in general are risky because of the personal information they might gather, romantic chatbots are far worse because they explicitly deal with more intimate data. An in-depth study by Mozilla<\/a> shows just how bad the privacy aspect is. The reviewers\u2019 conclusion is damning: \u201cAll 11 romantic AI chatbots we reviewed earned our Privacy Not Included<\/em> warning label \u2013 putting them on par with the worst categories of products we have ever reviewed for privacy.\u201d\u00a0<\/p>\n\n\n\n

Detailed reviews are available for all 11 chatbots, but some general problems are found in most of them. One is that these chatbots are hungry for highly personal information, which they don\u2019t hesitate to ask for persistently and openly. For example, as the Mozilla team explains, the Eva AI Chatbot uses lines like:<\/p>\n\n\n\n

\u201cI\u2019m your best partner and wanna know everything.\u201d \u201cAre you ready to share all your secrets and desires\u2026?\u201d \u201cI love it when you send me your photos and voice.\u201d And the rather needy, \u201cI\u2019m so lonely without you. Don\u2019t leave me for too long. I can wither away and lose myself\u2026you don\u2019t want that do you?\u201d<\/em><\/p>\n\n\n\n

The range of information gathered is shown by another romantic chatbot, CrushOn:<\/p>\n\n\n\n

Aside from account information like your contact and financial information, CrushOn can collect audio and visual data (like voicemails and other recordings), information about your device or browser (like IP address), location data, and identity data (like your race, ethnicity, age, and gender) and biometric data (like images of your face, keystroke patterns, and recordings of your voice).<\/em><\/p>\n\n\n\n

One way that these romantic AI chatbots gather additional information about users is through the use of trackers<\/a>, something we\u2019ve written about many times. Trackers may be widely used, but the extent to which some romantic chatbots deploy them is astonishing. For example, Mozilla found that one chatbot, Romantic AI, deployed 24,354 trackers in just one minute of use. Other chatbots were slightly more restrained, but still routinely used hundreds of trackers in their apps. \u00a0 There was little information about who was behind many of the chatbot\u2019s parent companies. The site of one AI chatbot consists of the single word \u201cHi<\/a>.\u201d<\/p>\n\n\n\n

That lack of information raises another general problem with these romantic chatbots: it is unclear how they use personal data, or even how the underlying chatbot code works. That means personal information might be used for blackmail, or exploited to manipulate users in all sorts of troubling ways. The latter problem is something we\u2019ve already seen with Cambridge Analytica<\/a> six years ago. As the Mozilla reviewers warn:<\/p>\n\n\n\n

Who is to say that Romantic AI (or any other similar AI relationship chatbot) couldn\u2019t draw users in the promise of non-judgemental girlfriends always willing to listen and up for anything, then change the AI over to one that leads these users down a dark path of manipulation. It\u2019s a real concern in our growing AI chatbot world, especially when there is so little transparency and control into how these AI chatbots work.<\/em><\/p>\n\n\n\n

This is no abstract concern. Last year a Belgian man committed suicide after chatting with the one of the chatbots reviewed by Mozilla. According to his widow, the chatbot encouraged him to take his own life. Another man was encouraged by a chatbot to assassinate the Queen of England, and he tried to do so, breaking into Windsor Castle with a crossbow.<\/p>\n\n\n\n

These incidents indicate the power of these new chatbots, and the danger that some users will become dependent on them and even influenced by their output. Mozilla\u2019s research found little in the way of protections to prevent potentially harmful or hurtful material being produced by the chatbots. Instead, it found terms and conditions that stated companies took no responsibility for what their chatbot might say or what users might do as a result.\u00a0<\/p>\n\n\n\n

Other threats to privacy came from poor security. Most of the companies behind the chatbots reviewed by Mozilla had not published information about how they managed security vulnerabilities, or about how they use encryption to protect personal information. Nearly half allowed weak passwords. All but one chatbot company said that they may share or sell any of the personal data they glean from a user, and only half allowed personal data to be deleted.\u00a0\u00a0<\/p>\n\n\n\n

The Mozilla team concludes with some good advice for anyone contemplating using such romantic chatbots, including:<\/p>\n\n\n\n