I. HALLUCINATING TRUTHS
Think of Open AI’s ChatGPT inferring an individual’s birth-date or bibliography incorrectly, Google’s Bard erroneously reporting a member of the Dutch Christian-Democratic party as running to be leader of another political party or more recent concerns voiced about the factual accuracy of DeepSeek’s answers. These are all instances of a widely reported problem common to all Large Language Models (LLMs), called hallucination: the LLM responds to a user prompt with plausible, yet non-factual, misleading or non-sensical data. Why? Essentially, LLMs generate text by identifying the most likely sequences of words seen during training but lackany true understanding of the text they produce. In other words, their ‘truth’ is the statistical one, not reality as we conceive it.
With this in mind, the question that is keeping many data protection experts busy is whether factually inaccurate personal data produced by LLMs – such as the ones presented in the examples above – are accurate under the GDPR and, if not, which measures the controller must take to rectify them. The data subjects who brought a case against Open AI because ChatGPT erroneously inferred their birth-date and bibliography invoked (among others) a violation of the principle of accuracy of Article 5.1 (d) GDPR and their right to rectification of Article 16 GDPR. Article 5.1.(d) GDPR requires personal data to be “accurate and up to date”; moreover, the controller should take “every reasonable step to ensure that personal data that are inaccurate, having regard to the purposes for which they are processed, are erased or rectified without delay”. According to Article 16 GDPR, “the data subject shall have the right to obtain from the controller without undue delay the rectification of inaccurate personal data concerning him or her”; additionally, “taking into account the purposes of the processing, the data subject shall have the right to have incomplete personal data completed, including by means of providing a supplementary statement”.
This blogpost will, first, deal with how accuracy should be assessed and what rectification means under the two aforementioned provisions. Second, it will explore what this implies for LLM generated personal data.
II. WHAT ARE ACCURATE PERSONAL DATA AND HOW SHOULD INACCURACIES BE RECTIFIED UNDER THE GDPR?
Below I will focus on four concepts that are frequently raised in discussions about accuracy and rectification of LLM generated personal data, namely: (i) purpose of the processing; (ii) factual accuracy; (iii) understanding of the data; (iv) rectification.
(i) Purpose of the processing
An analysis of the (very limited) case-law of the CJEU on the matter (i.e. Nowak case, para. 53) reveals that accuracy under Article 5.1.(d) GDPR’s predecessor, i.e. Article 6 (1)(d) of Directive 95/46/EC, should be evaluated in light of the purpose of the processing. A recent opinion by the Advocate General Collins in Deldits(para. 40) as well as recent EDPB Guidelines on processing personal data based on Article 6 (1) (f) GDPR(para.85) have confirmed this approach still stands. However, this so-called ‘purpose-test’ carries several challenges.
First, the exact meaning of this test is unclear. In Nowak this test implied that answers showing an exam candidate’s poor level of knowledge on the subject-matter were not inaccurate and could, hence, not be corrected a posteriori. While the Court did list several instances of inaccurate personal data in that case – e.g. the misattribution of the candidate’s answers to another candidate and a lack of correspondence between the evaluator’s actual comments and those reported on the exam sheet – these examples appear, from the language used by the Court, to be non-exhaustive (Nowak, para. 54). There are, hence, presumably other instances in which personal data would qualify as inaccurate in light of the purpose of the processing. However, due to scarce case-law and regulatory guidance on the topic, we can only speculate about what these instances might be.
Second, and linked to the first challenge, there are inconsistencies in the application of the purpose-test across national jurisdictions. For instance, the Belgian Court of Appeal (“Marktenhof”) ruled that a bank’s customer had the right to obtain rectification of an incorrectly spelled name, without referring to the purpose of the processing. In this case, the name processed by the bank did not contain an accent on the “e”, whereas the data subject’s name as it appeared on their ID-card, did. This sufficed for the Marktenhof to rule that the bank had to correct the name as requested by the data subject. In contrast, the Norwegian Privacy Appeals Board(“Personvernrådet”), judging on a similar case, held that the bank was not obliged to correct a customer’s name as requested by the latter. In this case, the individual sought correction of their prefix, from “Van” into “van”, as reported in their passport. The Personvernrådet evaluated the accuracy of the data in light of the purpose of the processing, i.e. administer the bank-customer relationship. According to the Board, the misspelling at issue did not prevent the bank from achieving such purpose, as there was no risk of misidentification of the data subject. Therefore, the misspelled name did not qualify as inaccurate personal data. While the differing outcomes may arguably stem from the existence of a risk of misidentification in the Belgian case, it would have been helpful if the Marktenhof had clarified this, which it did not.
(ii) Factual accuracy
The Belgian and Norwegian cases also show a third challenge raised by the purpose-test. While purpose-dependency implies a relative vision of accuracy, statements made in the past by the WP29, in its Guidance on the implementation of the Google Spain case C-131/12 (p. 15), and the EDPS, in its Guidelines on the Rights of Individuals with regard to the Processing of Personal Data (p. 18), refer to accuracy as a factual matter. This notion of factual accuracy appears to suggest that accuracy should be assessed regardless of the purpose of the processing. The aforementioned recent guidance by the EDPB seems to indicate, however, that these two approaches can be combined. While the purpose of the processing defines what is accurate, the term ‘factual’ (or ‘objective’) accuracy could relate, as also argued by other scholars, to the evidence required to substantiate the (in)accuracy.
(iii) Understanding of the data
An often overlooked aspect in current academic discussions on personal data accuracy is that the latter relates not only to the content of the data, but also to their understanding in a specific context. This appears clearly from early data protection scholarship. For instance, Page, who researched the right to rectification in early U.S. and Swiss data protection law in the 60’s – 80’s, states that personal data can be “be objectively false orproduce an incorrect image of the data subject in a specific context” (emphasis added) (See Page, Le droit d’accès et de contestation dans le traitement des données personnelles. Étude de base en droit privé suisse et américain, p. 299). Someone’s age, address and profession are examples of data that can be objectively false (See Page, p. 299). The incorrect image could be evoked by errors in the interpretation of the data, not necessarily errors in the data themselves. Such interpretation, Page continues, may be altered by a mismatchin the level of competences and/or interpretative criteria held by the author of the data, on the one hand, and the latter’s recipient, on the other hand. He provides the example of a statement describing a person as “nervous”: such statement may be accurate or not, depending on whether the author and the recipient of the statement share the same criteria concerning what makes a person nervous (See Page, p. 300). More recently, other data protection scholars touched upon the importance of human interpretation in relation to accuracy. Dimitrova, for instance, refers to “human cognition” as an important element determining personal data quality and stresses the importance of a “harmonized understanding” of personal data.
(iv) Rectification measures
What follows from the fact that accuracy – as a principle and as a data subject’s right – refers to the data andtheir understanding, is that the rectification measure the controller needs to adopt to remedy the inaccuracy depends on the inaccuracy at stake. In this regard, Page distinguishes correction from rectification. Broadly speaking, correction would concern errors in the (objectively verifiable) data only and would entail the erasure and, in certain cases, replacement of the data, when the controller failed to prove their accuracy (See Page, p. 306, 310). Rectification would pertain to errors in the understanding of the data. It would entail adding up data that are necessary to correct the global image that the data evoke about the individual in a specific context(See Page, p. 315). What would be added up, would only be data that are strictly necessary to rectify the said image (See Page, pp. 314-315). Dimitrova also offers an approach to rectification which is not limited to erasure or replacement of (objectively) inaccurate personal data but encompasses other measures, such as a correction of the presentation or format of the personal data, without altering the personal data themselves. Along similar lines, Drechsler, in her PhD thesis, also implicitly links rectification to the understanding of the data, and argues that rectification can imply “that it is made transparent that the data are not objective truths” (See Drechsler, Data subject rights in international personal data transfers, 2022, p. 50, unpublished).
Below, I will discuss how some Supervisory Authorities (SAs) have applied the aforementioned four concepts to LLM generated personal data and, subsequently, how I think these concepts should be applied to the case at hand.
III. WHAT ARE THE IMPLICATIONS FOR LLM GENERATED PERSONAL DATA?
(i) SAs’ stance so far: caught between purpose, factual accuracy and understanding of the data
The SAs that have engaged most extensively with the topic so far seem trapped in a cycle of circular reasoning. They tackle the matter from a purpose, factual accuracy, and misinterpretation perspective, yet they do not clearly answer how these impact the accuracy of LLM generated personal data and, especially, what should be done to rectify the inaccuracy.
When tackling the accuracy of ChatGPT output data, the EDBP’s ChatGPT Taskforce noted that the purposeof ChatGPT is “not to provide factually accurate information” but to “train ChatGPT” (para. 30). However, it also remarked that the answers provided by ChatGPT are “likely to be taken as factually accurate by end users […] regardless of their actual accuracy” (para. 30). Consequently, the Taskforce added, to avoid misinterpreting the personal data generated by ChatGPT, users should be sufficiently informed about ChatGPT’s probabilistic nature and limited level of reliability (para. 31). This would also follow from the principle of transparency of Article 5.1(a) GDPR. Although transparency measures may be beneficial to avoid misinterpretation of the output, the Taskforce concluded, back in May 2024, that they do not suffice to comply with the accuracy principle (para. 31).
Approximately 5 months later, when deciding upon the question whether ChatGPT complies with the GDPR, the Italian Data Protection Authority (“Garante”) re-iterated the aforementioned Taskforce’s considerations. The Garante added that, since its launch in November 2022, OpenAI has taken several measures to reduce the effects of inaccurate outputs, such as: (1) providing notices to users intended to avoid misinterpretation of ChatGPT’s output as factually accurate; (2) removing inaccuracies (e.g. through finetuning the model); (3) instructing the model not to provide users with private or sensitive data about individuals; and (4) allowing data subjects to signal the presence of inaccuracies in ChatGPT’s output and request their rectification. However, despite these measures, the Garante found that the problem of the inaccuracy of ChatGPT generated personal data was “far from being solved”. In other words, in November 2024, ChatGPT generated personal data were still inaccurate.
As regulators have yet to take a clear stance on the question of what accuracy and rectification imply for LLM generated personal data, below I offer my perspective on the topic.
(ii) Two rectification options left: sadly, both may be a dead end
I see two rectification options: either add up to the LLM generated personal data or correct them.
Adding up would entail targeting the understanding of each LLM’s answer, taking the average user in mind. However, this may not be commercially appealing to LLM providers. Scattered warnings about the limited factual accuracy of the LLM output and (inconsistent) refusals to reply to prompts containing personal data are already a step in the right direction, but, as also noted by the Garante in relation to ChatGPT, they do not conclusively solve the problem. The problem of misinterpretation of LLM output and potentially misleading image of an individual this may create is more fundamental, as it is likely to be inherent in the nature of the language used by LLMs. Specifically, LLMs are designed to produce coherent, fluent, well-structured and persuasive sentences, which give an aura of authoritativeness to their responses. As pointed out by Mittelstadt et al., if you couple this with “the human tendency to assign meaning and intent to […] words, misunderstanding is inevitable”. A recent study shows, indeed, that people tend to over-estimate the accuracy of LLM answers. The study concludes that people’s perception of accuracy could be improved, if the model clearly communicated uncertainty about the (factual) accuracy in each of its responses. With this, and other similar expert warnings in mind, any corrective measure that does not target the LLM’s response directly and clearly communicates uncertainty about the factual accuracy of the personal data contained in it may be unlikely to prevent users from perceiving ChatGPT’s answers as objectively true. While this measure would be indispensable for improving the perception of the LLM’s output, it would not be sufficient on its own. Additional transparency measures, such as, for instance, automatically providing sources with each response, would also be required. This being said, one may ask whether modifying the language of each answer into something more fallible, would not compromise the very essence of the LLM and, as a result, make it commercially unviable.
If altering the perception of LLM output data as factually correct is not a viable option, then the only other option left, in my opinion, is to treat the personal data generated by the LLM as facts and, consequently, rectify the data themselves. This means that the appropriate rectification measure will not be to add up to, but rather to correct (i.e. erase and/or replace) the inaccurate data. Yet, at the moment, this may be technically very challenging for controllers. In the past, OpenAI has indeed repeatedly invoked the technical limitations of LLMs as an argument to be exempted from personal data accuracy and rectification obligations. Particularly, the company argued that it is currently technically impossible for LLMs to always display responses that are factually accurate. Moreover, correcting inaccurate LLM output data (e.g. by finetuning the model) would, according to Open AI, not always be technically feasible. The likelihood of these arguments succeeding depends, in my opinion, on the specific GDPR accuracy provision at stake. Specifically, the accuracy provision of Article 5.1 (d) GDPR obliges the controller to take “reasonable steps” only to rectify or erase the inaccuracy. It has, hence, been interpreted as an obligation of means, not one of results (See De Bot, De toepassing van de Algemene Verordening Gegevensbescherming in de Belgische context, p. 497). By contrast, the right to rectification under Article 16 GDPR lacks any reference to reasonable steps. Therefore, accuracy under article 16 GDPR is arguably, an obligation of result, rather than means. The technical impossibility for LLMs to achieve a 100% accuracy rate could, hence, imply that an LLM’s display of factually inaccurate data is not per-se a violation of Article 5.1.(d) GDPR. However, the technical impossibility to replace inaccurate data with accurate one pursuant to a data subject’s request to rectification would not constitute a valid reason for refusing to correct the data as requested by the data subject. In other words, if the data subject objectively substantiates its correction request – e.g. provides a passport to attest his/her birthdate –, the controller would be required to correct such data. Technical impossibility could, at best, be used as an argument to erase the inaccurate personal data, instead of replacing them with accurate data.
To conclude, given the massive scale with which LLM-powered services are currently being offered and used, if altering the (perceived) accuracy of LLM-generated personal data does not appear to be commercially or technically viable, there is, in my opinion, only one solution left: prohibit the use of LLMs for generating personal data altogether.
Stephanie Rossello is a Doctoral candidate at Open Universiteit (Heerlen, The Netherlands) and KU Leuven (Leuven, Belgium). Her research focuses on the right to rectification of inaccurate personal data in general and, in particular, applied to AI inferences and AI systems. Prior to starting her PhD, Stephanie was working as a researcher at the Center for IT and IP Law (KU Leuven), and as inhouse counsel and lawyer specializing in EU data protection, anti-trust and real-estate law.
