Publications

Introduction On 21 May 2024, the Council of the European Union adopted the Regulation laying down harmonised rules on artificial intelligence” (the so-called AI Act). As the world's first comprehensive law to regulate artificial intelligence, the AI Act aims to establish uniform requirements for the development and use of artificial intelligence in the European Union. Following the European Parliament's adoption of the draft on 13 March 2024, the AI Act has now been formally adopted . Once signed by the Presidents of the European Parliament and the Council, the Regulation will be published in the Official Journal of the EU and will enter into force twenty days after its publication. With this  adoption of the world’s most significant legislation on Artificial Intelligence, solidifying its position as a pioneer among global legislators. This initiative aims to establish and reinforce the EU’s role as a premier hub for AI while ensuring that AI development remains focused on human-centered and trustworthy principles. After a long and complex journey that began in 2021 with the European Commission’s proposal of a draft AI Act, this new regulation is expected to be passed into law in June  2024. The AI Act aims to ensure that the marketing and use of AI systems and their outputs in the EU are consistent with fundamental rights under EU law, such as privacy, democracy, the rule of law and environmental sustainability. Adopting a dual approach, it outright prohibits AI systems deemed to pose unacceptable risks while imposing regulatory obligations on other AI systems and their outputs. The new regulation, which also aims to strike a fair balance between innovation and the protection of individuals, not only makes Europe a world leader in the regulation of this new technology, but also endeavours to create a legal framework that users of AI technologies will be able to comply with in order to make the most of this significant development opportunity. In this article we provide a first overview of the key points contained in the text of the AI Act that companies should be aware of in order to prepare for the implementing regulation.

Following the European Parliament's approval of the draft on 13 March 2024 , the Council of the European Union has also approved the final text of the AI Act on 21 May 2024, meaning that the AI Act has been formally adopted. The AI Act will enter into force on the 20th day after publication in the EU Official Journal and will be applicable after 24 months. However, some specific provisions will have different application dates, such as prohibitions on AI, that will apply 6 months after entry into force; or General Purpose AI models already on the market, which are given a compliance deadline of 12 months. The AI Office was established on 21 February 2024 and the European Commission will oversee the issuance of at least 20 delegated acts. The AI Act’s implementation will be supported by an expert group formed to advise and assist the European Commission in avoiding overlaps with other EU regulations. Meanwhile, Member States must appoint at least one notifying authority and one market surveillance authority and communicate to the European Commission the identity of the competent authorities and the single point of contact. The next regulatory step appears to be focused on AI liability. On 14 December 2023, EU policymakers reached a political agreement on the amendment of the Product Liability Directive. This proposal aims to accommodate technological developments, notably covering digital products like software, including AI. The next proposal in line in the AI package is the Directive on the ad­apt­a­tion/har­mon­iz­a­tion of the rules on non-contractual civil liability to Artificial Intelligence (AI Liability Directive). Addressing issues of causality and fault related to AI systems, this directive proposal ensures that claimants can enforce appropriate remedies when suffering damages in fault-based scenarios. The draft was published on 28 September 2022 and is still pending to be considered by the European Parliament and Council of the European Union . Once adopted, EU Member States will be obliged to transpose its provisions into national law within a likely two-year timeframe. The enactment of the AI Act represents a pivotal step towards fostering a regulatory landscape, not only in the EU but worldwide, that balances innovation, trust, and accountability, ensuring that AI serves as driver of progress while safeguarding fundamental rights and societal values.

Codes of conduct (Chapter X, Art. 95)In order to foster ethical and reliable AI systems and to increase AI literacy among those involved in the development, operation and use of AI, the new AI Act mandates the AI Office and Member States to promote the development of codes of conduct for non-high-risk AI systems. These codes of conduct, which should take into account available technical solutions and industry best practices, would promote voluntary compliance with some or all of the mandatory requirements that apply to high-risk AI systems. Such voluntary guidelines should be consistent with the EU values and fundamental rights and address issues such as transparency, accountability, fairness, privacy and data governance, and human oversight. Furthermore, to be effective, such codes of conduct should be based on clear objectives and key performance indicators to measure the achievement of these objectives. Codes of conduct may be developed by individual AI system providers, deployers, or organizations representing them and should be developed in an inclusive manner, involving relevant stakeholders such as business and civil society organisations, academia, etc. The  European Commission will assess the impact and effectiveness of the codes of conduct within two years of the AI Act entering into application, and every three years thereafter. The aim is to encourage the application of requirements for high-risk AI systems to non-high-risk AI systems, and possibly other additional requirements for such AI systems (including in relation to environmental sustainability).

Governance (Chapter VII, Art. 64-70 )The AI Act establishes a governance framework under Chapter VII, with the scope of coordinating and supporting its application on a national level, as well as build capabilities at Union level and integrate stakeholders in the field of artificial intelligence. The measures related to governance will apply from 12 months following the entry into force of the AI Act. To develop Union expertise and capabilities, an AI Office is established within the Commission, having a strong link with the scientific community to support its work which includes the issuance of guidance; its establishment should not affect the powers and competences of national competent authorities, and bodies, offices and agencies of the Union in the supervision of AI systems. The newly proposed AI governance structure also includes the establishment of the European AI Board (AI Board), composed of one representative per Member State, designated for a period of 3 years. Its list of tasks has been extended and includes the collection and sharing of technical and regulatory expertise and best practices in the Member States, contributing to their harmonisation, and the assistance to the AI Office for the establishment and development of regulatory sandboxes with national authorities. Upon request of the Commission, the AI Board will issue recommendations and written opinions on any matter related to the implementation of the AI Act. The Board shall establish two standing sub-groups to provide a platform for cooperation and exchange among market surveillance authorities and notifying authorities on issues related to market surveillance and notified bodies. The final text of the AI Act also introduces two new advisory bodies. An advisory forum (Art. 67) will be established to provide stakeholder input to the European Commission and the AI Board preparing opinions, recommendations and written contributions.A scientific panel of independent experts (Art. 68) selected by the European Commission will provide technical advice and input to the AI Office and market surveillance authorities. The scientific panel will also be able to alert the AI Office of possible systemic risks at Union level. Member States may call upon experts of the scientific panel to support their enforcement activities under the AI Act and may be required to pay fees for the advice and support by the experts. Each Member State shall establish or designate at least one notifying authority and at least one market surveillance authority as national competent authorities for the purpose of the AI Act. Member States shall ensure that the national competent authority is provided with adequate technical, financial and human resources and infrastructure to fulfil their tasks effectively under this regulation, and satisfies an adequate level of cybersecurity measures. One market surveillance authority shall also be appointed by Member States to act as a single point of contact.

General purpose AI models (Chapter V  V, Art. 51-56)The AI Act is founded on a risk based approach. This regulation, intended to be durable, initially wasn’t associated to the characteristics of any particular model or system, but to the risk associated with its intended use. This was the approach when the proposal of the AI Act was drafted and adopted by the European Commission on 22 April, 2021, when the proposal was discussed at the  Council of the European Union on 6 December, 2022. However, after the great global and historical success of generative AI tools in the months following the Commission’s proposal, the idea of regulating AI focusing only on its intended use seemed then insufficient. Then, in the 14 June 2023 draft, the concept of “foundation models” (much broader than generative AI) was introduced with associated regulation. During the negotiations in December 2023, some additional proposals were introduced regarding “very capable foundation models” and “general purpose AI systems built on foundation models and used at scale”. In the final version of the AI Act, there is no reference to “foundation models”, and instead the concept of “general purpose AI models and systems” was adopted. General Purpose AI models (Arts. 51 to 56 ) are distinguished from general purpose AI systems (Arts. 25 and 75). The General Purpose AI systems are based on General Purpose AI models: “when a general purpose AI model is integrated into or forms part of an AI system, this system should be considered a general purpose AI system” if it has the capability to serve a variety of purposes (Recital 100). And, of course, General Purpose AI models are the result of the operation of AI systems that created them.“General purpose AI model” is defined in Article 3 (63) as “an AI model (…) that displays significant generality and is capable to competently perform a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications”. The definition lacks quality (a model is “general purpose” if it “displays gen­er­al­ity”1Re­cit­al 98 contributes to clarify the concept saying that “generality” means the use of at least a billion of parameters, when the training of the model uses “a large amount of data using self-supervision at scale”. footnote) and has a remarkable capacity for expansion. Large generative AI models are an example of General Purpose AI models (Recital 99). The obligations imposed to providers of General Purpose AI models are limited, provided that they don’t have systemic risk. Such obligations include (Art. 53 (I) (a)  (i) to draw up and keep up-to-date technical documentation (as described in Annex XI) available to the national competent authorities, as well as to providers of AI systems who intend to integrate the General Purpose AI system in their AI systems, and (ii) to take some measures in order to respect EU copyright legislation, namely to put in place a policy to identify reservations of rights and to make publicly available a sufficiently detailed summary about the content used. Furthermore, they should have an authorised representative in the EU (Art. 54). The most important obligations are imposed in Article 55 to providers of General Purpose AI models with systemic risk. The definition of AI models with systemic risk is established in Article 55 in too broad and unsatisfactory terms: “high impact capabilities”. Fortunately, there is a presumption in Article 55.2 that helps: “when the cumulative amount of compute used for its training measured in floating point operations (FLOPs) is greater than 10^25”. The main additional obligations imposed to General Purpose AI models with systemic risks are (i) to perform model evaluation (including adversarial testing), (ii) to assess and mitigate systemic risks at EU level, (iii), to document and report serious incidents and corrective measures, and (iv) to ensure an adequate level of cybersecurity. Finally, an “AI system” is “an AI system which is based on a General Purpose AI model, that has the capacity to serve a variety of purposes” (Art. 3 (66)). If General Purpose AI systems can be used directly by deployers for at least one purpose that is classified as high-risk (Art. 75), an evaluation of compliance will need to be done, if there is sufficient reason to consider that the system is not compliant with AI Act.

Prohibited Artificial Intelligence practices (Chapter II, Art. 5) 1. Introduction to the unacceptable risk category Article 5 categorises certain AI technologies as posing an “unacceptable risk” (Unacceptable Risk). Unlike other risk categories outlined in the AI Act, the use of AI technologies that fall within this category is strictly prohibited ("Prohibited AI Systems"). It is therefore necessary to distinguish between:those technologies that are clearly prohibited; andthose AI applications that are not clearly prohibited but may involve similar risks. The most challenging problem in practice is to ensure that activities, which are not prohibited, do not become Unacceptable Risk activities and therefore prohibited. 2. Unacceptable Risk: Prohibited AI practices Article 5 explicitly bans harmful AI practices: The first prohibition under Article 5 addresses systems that manipulate individuals or exploit their vulnerabilities, leading to physical or psychological harm. Accordingly, it would be prohibited to place on the market, put into services or use in the EU:AI systems designed to deceive, coerce or influence human behaviour in harmful ways; andAI tools that prey on an individual’s weaknesses, exacerbating their vulnerabilities. The second prohibition covers AI systems that exploit these vulnerabilities, even if harm is not immediate. Examples include:AI tools that compromise user privacy by collecting sensitive data without consent; andAI algorithms that perpetuate bias or discrimination against certain groups. The third prohibition focuses on the use of AI for social scoring. Social scoring systems assign scores to individuals based on their behaviour, affecting access to services, employment or other opportunities. Prohibited practices in­clude:AI-driv­en scoring mechanisms that lack transparency, fairness or accountability; andSystems that discriminate based on protected characteristics (e.g. race, gender, religion). The fourth prohibition covers biometric real-time identification in publicly accessible spaces for law enforcement purposes. This includes:AI systems that identify individuals without their knowledge or consent; andContinuous monitoring of people’s movements using biometric data. 3. Clearly listed: Best practices and compliance Transparency and accountability are essential in complying with the prohibitions under Article 5. Firms using AI must design and continuously test systems, be transparent about their intensions and avoid manipulative practices. They should also disclose AI systems functionality, data usage, and decision-making processes. Companies should conduct thorough impact assessments to identify unintended vulnerabilities and implement specific safeguards to prevent exploitation. This should form part of assessments of AI systems to understand their impact on individuals and society. Companies should develop clear guidelines for scoring systems to prevent the development of social scoring characteristics, and prioritise ethical design, fairness and non-dis­crim­in­a­tion. Privacy impact assessments should be pursued to ensure compliance with the various prohibitions. In particular, firms should be very careful using any real-time identification systems. In all cases, companies should maintain comprehensive records of AI system design, training, and deployment. Any critical decision made by AI systems should be overseen by a human. 4. Not clearly listed: Categorisation Unacceptable Risk AI systems cover systems that are deemed inherently harmful and are considered a threat to human safety, livelihoods, and rights In contrast, high-risk AI systems cover systems designed to be applied to specific use cases, including using AI for hiring and recruitment that may cause harm but are not inherently harmful. High risk AI systems are legal, but subject to important requirements under the AI Act. It is therefore crucial to determine the difference between high risk and unacceptable risk AI systems. In essence, any high risk activity can escalate to Unacceptable Risk under the following cir­cum­stances:Bi­as and Discrimination: if AI perpetuates bias or discriminates against protected groups. Privacy Violations: when AI systems compromise user privacy or misuse sensitive data. Psychological Harm: if AI manipulates individuals, causing psychological distress. AI systems that are able to perform generally applicable functions and are able to have multiple intended and unintended purposes (being General Purpose AI models) are not inherently prohibited under the AI Act, but must be used with care since in certain scenarios they lead to Unacceptable Risk activities. To assess whether a General Purpose AI Model poses an Unacceptable Risk, it is necessary to consider the context in which the model operates. If it influences critical decisions (e.g. hiring, credit scoring), perpetuates bias or discriminates, compromises user privacy (e.g. by collecting sensitive data without consent), the risk increases, and the model may need to be adapted. 5. Best practice and compliance While the AI Act provides examples of explicit prohibitions under the AI Act, it cannot cover all possible situations as the technology is, through updated versions and by definition, constantly evolving. As a guide, legal and compliance teams should ask the following questions when considering high- risk AI systems:Risk assessment:What is the evidence that the categorisation of the AI application is minimal, limited, high or Unacceptable Risk?Does the application in any circumstances use or act on sensitive data or influence critical de­cisions?Con­tex­tu­al analysis:Does the application operate in a sector that has a presumption of increased risk, for example, (a) financial services, or (b) healthcare?In what ways does the deployment of the application impact (a) individuals, and (b) society?Specific criteria:Can any decisions of the application be considered to give rise to manipulation, exploitation, discriminatory scoring, or biometric iden­ti­fic­a­tion?Does the application operate or have access to data that could give rise to the exploitation of subliminal techniques or vulnerabilities related to protracted characteristics, such as age or dis­ab­il­ity?Trans­par­ency and Documentation:In what ways is the AI system transparent about its inherent functioning and de­cision-mak­ing?In what ways does the user’s documentation of the design, training and deployment of the application demonstrate compliance with the various rules? 6. Conclusion Unacceptable Risk AI activities are those practices that pose inherent harm to people and are strictly forbidden under the AI Act. The potential for reputational damage and regulatory sanctions serve as strong deterrents for firms to avoid breaching these provisions of the AI Act. It is essential for companies to take proactive measures to ensure compliance and prevent harm to individuals and society.