The Italian data protection authority has imposed a fine of €5 million on the US company Luka, Inc., operator of the AI-based chatbot ‘Replika’, and launched an independent investigation into the processing of personal data by the AI system.
Replika is a chatbot that serves as a virtual companion for emotional support and can act as a friend, therapist, romantic partner or mentor, depending on the user’s wishes. The Italian data protection authority has made the following data protection allegations against Luka, Inc.
- Lack of evidence of appropriate legal basis (Art. 6 GDPR):
The privacy policy did not specify the legal basis for individual processing operations, in particular for the further processing of data for training purposes of the LLM.
- Inadequate privacy policy (Art. 13 and 14 GDPR):
The privacy policy, which was only available in English and was incomplete, was neither understandable for Italian-speaking users nor transparent with regard to key aspects of data processing, such as storage duration, third-country transfers and automated decision-making.
- Lack of age verification (Art. 8 GDPR):
Luka did not implement effective technical measures for age verification, even though they excluded minors from using the service. Despite clearly stating that they were under 18, minors had unrestricted access to age-inappropriate content when using Replika.
The Replika case makes it clear that AI applications must be designed to be privacy-compliant from the outset, especially if they have an emotional or psychological impact on users. Transparency in automated decisions, protection of minors and clear purpose limitation of data processing are essential. The case also shows that supervisory authorities also examine complex AI systems and consistently sanction breaches.
We would be happy to support you with a legal review or strategic consultation on the GDPR-compliant design of your AI applications. Get in touch with us!