What Zoom’s AI Blunder Can Teach Us About Future Threats to Privacy
It’s no surprise that AI is starting to seep into every kind of digital service, even if the frenzy over generative AI seems to be dying down somewhat. But over on Stackdiary, Alex Ivanovs spotted something rather remarkable in Zoom’s updated terms and conditions that involved AI. The new wording seemed to give Zoom the right to treat content generated by its users as training for AI programs, and with no way to opt out. The new terms required users to grant Zoom “a perpetual, worldwide, non-exclusive, royalty-free, sublicensable, and transferable” license for a number of purposes, including:
product and service development, marketing, analytics, quality assurance, machine learning, artificial intelligence, training, testing, improvement of the Services, Software, or Zoom’s other products, services, and software, or any combination thereof;
As a previous PIA blog post explained back in February, training AI on this kind of material carries major privacy risks. These issues are now more widely understood, and when people found out about Zoom’s plans, many were outraged. As Ivanovs explains in an update to his initial post — Zoom, now confronted by this reaction, engaged in some hasty damage limitation. First, it published a blog post explaining why it had made changes to the terms and conditions. In addition, it inserted the following line into the section mentioning AI training: “Notwithstanding the above, Zoom will not use audio, video, or chat Customer Content to train our artificial intelligence models without your consent.”
However, as many were quick to point out, this changed very little since people were essentially offered the choice of accepting the possible use of their sessions for AI training or not accessing Zoom. That’s not just problematic, but probably illegal in the EU, where the courts have said such “forced consent” is not permissible under the GDPR. The CEO of Zoom, Eric Yuan, wrote a LinkedIn post, in which he blamed a “process failure internally” for the previous inclusion of AI training. More significantly, he wrote:
We are commiting to all of our customers that we will not use any of their audio, video, chat, screen sharing, attachments and other communications like poll results, whiteboard, and reactions to train our AI models or third-party AI models.
That commitment was indeed incorporated in a newer version of Zoom’s terms and conditions. Those terms were drastically simplified compared to previous texts, with most of the troubling references to AI training removed. However, a new term has been added, which reads as follows:
Telemetry data, product usage data, diagnostic data, and similar data that Zoom collects or generates in connection with your or your End Users’ use of the Services or Software are referred to as Service Generated Data. Zoom owns all rights, title, and interest in and to Service Generated Data.
Zoom’s “telemetry data” is what most people call metadata: that is, ancillary data describing the main data, in this case, generated by customers during Zoom calls. Zoom not only asserts its unequivocal ownership of this metadata, but it makes a clear distinction between metadata and customer-generated data. As a result, its commitment not to use customer data to train AI systems does not apply to this data.
As PIA blog readers know, metadata can be just as revealing about people and their activities as the main data. If Zoom uses this metadata to train its AI systems, it could easily also use techniques like data aggregation to extract highly personal and confidential information.
Another potential issue is that the company’s latest terms and conditions state “Zoom does not use any of your audio, video, chat, screen sharing, attachments or other communications-like Customer Content (such as poll results, whiteboard, and reactions) to train Zoom or third-party artificial intelligence models.” That commitment seems to refer to present activities (“does not”), not future ones (“will not”).
Moreover, Section 15 of the terms and conditions requires users to “agree that Zoom may modify, delete, and make additions to its guides, statements, policies, and notices, with or without notice to you.” That means Zoom could change the terms to allow the use of customer-generated material for AI training at some point in the future.
One problem is that Zoom doesn’t have the best track record on transparency. The company has been implicated in numerous security and privacy issues over the years. More recently, Zoom settled with the US Federal Trade Commission over allegations that the company “misled users by touting that it offered ‘end-to-end, 256-bit encryption to secure users’ communications,’ when in fact it provided a lower level of security.”
Another, more general, problem is that if Zoom refrains from using customer data to train its AI systems now and in the future, it seems highly likely that other companies will not. It will probably be difficult in the EU, but elsewhere the potential rewards from mining customer data in this way may outweigh the legal and financial risks of doing so. If nothing else, Zoom’s recent AI missteps have provided a timely warning of a new threat to privacy that people should be aware of.
Featured image by Zoom.
