The risk of discrimination highlighted by these researchers became painfully real in the Netherlands. In 2013, the Dutch Tax Authorities employed a self-learning algorithm to detect child care benefits fraud. The risk indicators used by the system included having low income or belonging to ethnic minorities. As a result, thousands of families were wrongly characterized as fraudsters and suffered severe consequences. This led to the Dutch Government’s resignation and a 3.7 million Euros fine on the Tax Administration from the Autoriteit Persoonsgegevens, the Dutch Data Protection Authority, for breaching several GDPR rules. In particular, the Authority found that the Tax Administration had no legal basis for processing the personal data used as risk indicators. Under the GDPR, personal data processing is allowed only if one of the legal bases listed in Article 6 applies.

In the hyper-regulated European Union, the GDPR has attempted to address these issues by introducing Article 22, which allows individuals to opt out of "automated decision making, including profiling, and obtain human intervention whenever their personal information is used to take a decision which produces a legal effect (e.g., entering into a contract with that individual). Additionally, the proposed EU AI Act aims to place serious limitations on "AI systems providing social scoring of natural persons for general purposes by public authorities." These limitations prohibit social scoring systems from leading to detrimental or unfair treatment in unrelated social contexts or based on unjustified or disproportionate criteria.