The EU AI Act Could Be the Next GDPR – The EU Parliament Wants Strong Privacy Protections

In a key vote, the European Parliament has agreed its negotiating position for the finalization of the forthcoming EU Artificial Intelligence (AI) Act. The proposed text contains many important protections for privacy, including bans on the following:

• “Real-time” remote biometric identification systems in publicly-accessible places
• Biometric identification post-processing of captured data, with an exception for prosecution of serious crimes and with a judge’s approval
• Biometric categorization systems that employ sensitive characteristics such as gender, race, ethnicity, religion and political views
• Predictive policing systems, typically based on profiling, location or previous criminal behavior
• Emotion recognition systems in law enforcement, border management, the workplace and educational establishments – however, companies can use this technology on consumers
• Untargeted scraping of facial images from the Internet or CCTV to create facial recognition databases such as Clearview AI, previously discussed on our blog.

Those proposed protections represent a big win for privacy. As we reported back in 2021, there was doubt whether many of them would be included, and the fact that they made it into the Act represents, at least in part, a growing awareness that increasingly-powerful AI systems pose a threat to everyone’s privacy.

Most LLMs Don’t Comply with the EU AI Act

The latest generative AI systems are a privacy disaster waiting to happen. Fortunately, the European Parliament’s text now includes measures to address that danger. The press release explains:

Providers of foundation models – a new and fast-evolving development in the field of AI – would have to assess and mitigate possible risks (to health, safety, fundamental rights, the environment, democracy and rule of law) and register their models in the EU database before their release on the EU market. Generative AI systems based on such models, like ChatGPT, would have to comply with transparency requirements (disclosing that the content was AI-generated, also helping distinguish so-called deep-fake images from real ones) and ensure safeguards against generating illegal content.

An analysis by the Stanford Center for Research on Foundation Models found that most LLMs don’t comply with the EU’s AI Act in its current form. The researchers noted that open models scored better than restricted or closed ones. Open approaches received an important boost in the European Parliament text, which added exemptions for research activities and AI components provided under open-source licenses, something welcomed by the Free Software Foundation Europe.

At the heart of the EU’s AI Act is a “risk-based” approach. AI systems deemed a “clear threat” to the safety, livelihood and people’s rights will be banned. So-called “high-risk” AI systems will be subject to a variety of obligations before they can be put on the market, including:

• risk assessment
• high quality datasets
• logging of activity
• detailed documentation
• human oversight
• high level of robustness, security, and accuracy

The European Parliament’s list of high-risk AIs also included systems used to influence voters and the outcome of elections, and those in recommendation systems used by social media platforms (with over 45 million users).

Businesses Using AI Can Regulate Themselves

Despite all the good news, the European consumer organisation BEUC pointed out that the current text gives businesses the option to decide for themselves whether their AI system should be treated as high risk or not, which creates a rather large AI loophole.

The European digital rights group EDRi was pleased with what it termed “a win for fundamental rights”. However, it criticized what is saw as a failure to introduce new provisions that would protect the rights of migrants regarding AI-based surveillance, which is increasingly being deployed to track, control and monitor migrants.

In theory, the missing elements could be added during the so-called “trilogue” negotiations that are now taking place between the European Parliament, the European Commission, and the Council of the European Union. That said, it’s much more likely that the European Commission, and the Council of the European Union will seek to water down some of the privacy protections found in the European Parliament’s text. This means that efforts to safeguard privacy in the coming world of AI systems are far from over.

This is a crucial battle, because the EU’s AI Act is by far the most advanced and ambitious attempt at a legislative framework for regulating AI technologies. In this respect, it is potentially as important and far-reaching in its impact as the EU’s General Data Protection Regulation (GDPR), which was passed in 2016. As many stories on PIA blog attest, the GDPR has exerted its influence globally, acting as the model for similar privacy laws far beyond the EU. It has also forced many of the most powerful US-based Internet companies to modify their business practices to comply with the GDPR.

The US Develops Tech, the EU Regulates It

Assuming that the trilogue negotiations don’t completely undermine the protections currently found the European Parliament’s proposed text, it seems likely that the EU AI Act will set the standard for legislative moves in this area. For example, a recent editorial in the leading journal Science called the draft version “An EU landmark for AI governance”, and wrote that the AI Act “provides a glimpse into the future of governance of artificial intelligence”.

Against that background, the recent US initiative to “Promote Responsible AI Innovation that Protects Americans’ Rights and Safety”, and a new proposal for a “blue ribbon” commission to develop a strategy for regulating AI, are rather underwhelming. Once more, it looks like the US will dominate many of the key technologies underlying the latest exciting AI developments, but it will be the EU that sets the legal rules governing their use.

Featured image by Hann Lans.