AI Systems for Social Scoring

Article 5(1)(c) of the eu-ai-act-regulation-2024-1689 prohibits "the placing on the market, the putting into service or the use of AI systems for the evaluation or classification of natural persons or groups of persons over a certain period of time based on their social behaviour or known, inferred or predicted personal or personality characteristics, with the social score leading to either or both of the following:

(i) detrimental or unfavourable treatment of certain natural persons or groups of persons in social contexts that are unrelated to the contexts in which the data was originally generated or collected;

(ii) detrimental or unfavourable treatment of certain natural persons or groups of persons that is unjustified or disproportionate to their social behaviour or its gravity."

Key Elements of Prohibition

Evaluation/Classification: Systems that assess or categorize people based on:

• Social behavior patterns
• Known personal characteristics
• Inferred personality traits
• Predicted characteristics

Temporal Element: Assessment occurs "over a certain period of time," indicating ongoing monitoring and evaluation.

Harmful Outcomes: The social score leads to:

• Treatment in unrelated contexts (cross-context discrimination)
• Unjustified or disproportionate treatment

Rationale (Recital 31)

Social scoring systems:

• May lead to discriminatory outcomes
• May exclude certain groups
• May violate dignity and non-discrimination rights
• May violate values of equality and justice
• May result in detrimental treatment disproportionate to behavior

Scope of Prohibition

The prohibition covers:

• Public and private actors
• Any social scoring system meeting the criteria
• Both direct and indirect discriminatory effects

Exceptions

The prohibition "shall not affect lawful evaluation practices of natural persons that are carried out for a specific purpose in accordance with Union and national law."

This preserves legitimate evaluation systems such as:

• Credit scoring for specific financial purposes
• Performance evaluations in employment contexts
• Academic assessments
• Professional licensing evaluations

Provided these are:

• Conducted for specific, lawful purposes
• Compliant with existing Union and national law
• Proportionate to their specific context

Enforcement

Violation of this prohibition is subject to administrative fines of up to €35 million or 7% of global annual turnover, whichever is higher.