Main principles of the AI Act

^ Applicability and exemptions

EU and non-EU providers

The AI Act does not apply only to EU providers. It regulates products that circulate in EU, not those that are produced in EU. The obligations of the AI Act apply to all AI systems and AI models that are placed on the EU market or put into service in EU, regardless of whether the provider is in the EU or out of the EU.

This means that a non-EU provider who wants to enter the EU market needs to comply with the same obligations as any EU provider. This does not create unfair competition from non-EU providers over the EU ones.

It also applies to AI systems that are physically outside of the EU but whose output is used in the EU. For instance, an online platform whose servers are out of the EU is not exempted if it is used by people in the EU.

Non-EU providers (based out of the EU and with no subsidiary in the EU) are expected to appoint an authorised representative within the EU in order to act as the main point of contact for legal procedures, including checking that the provider has complied with their obligations (technical documentation, conformity assessment…) and providing any necessary information to authorities. This applies both to providers of high-risk AI systems (Article 22) and to providers of General-Purpose AI models (Article 54).

Military, defence or national security purposes

AI systems used solely for these purposes are exempted, including when developed or used by a private organisation.

This does not mean that all AI systems related to the military are exempted, it depends on the purpose: for instance, AI systems used for employment in the military are not exempted.

Dual-use systems are not exempted.

Scientific research and development

AI systems and AI models developed and put into service for the sole purpose of scientific research and development are exempted. This includes:

  • AI used to support research in other disciplines: for instance, an AI system that analyses experimental data for a genomics study is exempted.

  • Research on AI: for instance, if a PhD student in AI publishes a paper and releases the corresponding AI model to facilitate reproducibility of the work and continuation of the research, that AI model is exempted.

This does not mean that any AI system or AI model developed by a researcher is exempted:

  • If an AI researcher releases an AI model publicly so that it can be used by others (including companies) to deploy it or integrate it into a product, then the AI model is not exempted. This applies for instance to models released on Hugging Face.

  • If an AI researcher develops an AI system and puts it online on their website as a demo, but it can effectively be used there by others for its own purpose, then that AI system is not exempted.

This does not mean either that all AI systems used in a research institute are exempted. For instance, an AI system used by the human resources of a research institute is not exempted.

The AI Act does not apply to R&D work conducted in preparation for a future AI system, when the provider is still experimenting. However, for the actual AI system that is placed on the market in the end, the entire development of that particular AI system has to comply with the AI Act obligations (including the training of any machine learning model inside it, or the preparation of its datasets), even if some steps have been conducted during the R&D phase.

Free and open-source licences

AI systems released under a free and open-source licence are not fully exempted but they benefit from reduced obligations. In particular, obligations that apply to all AI systems, such as the AI literacy one, do not apply by default on AI systems released under a free and open-source licence. But as soon as these AI systems fall under the prohibited (Article 5), high-risk (Article 6), or transparency case (Article 50), they remain fully subject to all AI Act obligations, regardless of their licence.

General-Purpose AI models (without systemic risks) that are released under a free and open-source licence are exempted from the documentation obligations for GPAI models towards authorities and towards downstream providers (considering that this licence already offers sufficient transparency), but they still have to comply with obligations related to copyright and to the training data summary. Non-EU providers of these models do not need an authorised representative in the EU.

An open-weight model is not necessarily considered open-source. Being open-source implies for instance the ability to modify, which is stronger than just having access to the model weights. It also includes expectations on the freedom to use the model, for instance the exemption does not apply to models that are limited to non-commercial use.

See also:

^ The definition of AI systems

The challenge of defining AI

While it is obvious that certain systems are AI systems, and obvious that others are not, delineating the precise boundary is known to be highly challenging. Some cases are debatable and subject to interpretation. Some cases are perceived differently by different people depending on their background or experience.

The European Commission has clearly acknowledged that no definite list of what is or is not an AI system can be crafted, and it will always remain a per-case decision. That assessment can however rely on a number of criteria that are established in the AI Act, through the legal definition of ‘AI system’ in Article 3(1).

Criteria to identify AI systems

An AI system is computationally driven: it is developed with and runs on machines, it is based on machine operations (in a broad sense, including for instance quantum computing).

At the core of the AI system are the objectives it pursues. An AI system does not just perform operations, but performs them to achieve a goal: the task or tasks that it has been designed for. These objectives (e.g. answering questions about documents), which pertain to the system itself, are more specific than the intended purpose (e.g. improving a team’s functioning by handling a set of documents faster), which pertains to how the AI system is used in its context, and also includes for instance the consideration of the domain in which this task is to be performed, the characteristics of the targeted users or how the AI system can be integrated into a broader workflow.

Once deployed, an AI system is connected to an environment, in which it can act to influence that environment. But this environment is not necessarily physical (as in robotics) and can be a virtual environment, such as a digital space (a social media platform, or any other platform with an API), but also a software ecosystem, or just a data flow. Creating a new file in a folder, or displaying an answer on a screen, are also ways to influence the environment.

An AI system is expected to have some degree of independence of action from human involvement and to operate without human intervention: it is not a mere tool that the human has to manipulate at each step, like a screwdriver. This idea is called ‘autonomy’ in the AI Act, but in a broad sense that does not imply that the AI system can go beyond its instructions, nor that the human has necessarily no clear idea of what is happening inside the machine. It refers more to the fact that the human does not intervene from the moment the AI system receives its input to when it returns its output, and this usually complex process (the inference) unfolds solely based on the way the AI system has been designed (sometimes explicitly, sometimes less so).

The legal definition of an AI system accounts for both the ‘building’ phase, during which certain technical processes can operate on or within the AI system (which can include machine learning, but not necessarily) to make it able to perform its future operation, and the ‘use’ phase, during which the AI system produces its actual outputs for its intended purpose. These two phases can be interleaved, by simulating use for building purposes, or pursuing the building by leveraging information acquired through its use. Some parts of the AI system can be involved in only one of these two phases.

Finally, AI systems are characterised by the nuances in their outputs, or the nuances in the way to produce these outputs: this typically involves complex relationships, patterns, or some reasoning mechanisms. For instance, a system taking a sentence and putting it into uppercase would not constitute an AI system, but a system that takes a sentence and rephrases it would be considered as an AI system.

The diversity of AI systems

While certain types of AI systems, such as chatbots, are much better known by the public, AI systems encompass in practice a wide spectrum of diverse systems, use cases and technologies.

The AI Act targets both the AI systems that are used on a stand-alone basis and those that are integrated as a component into a broader product. This includes both physical integration (case of an embedded AI system) and pure software integration (as part of a processing pipeline for instance), as well as cases where the AI system itself is not located within the boundaries of the product (such as an AI system deployed in the cloud to serve a functionality that is necessary for the functioning of a physical product in the consumer’s home).

The objectives that are pivotal to what constitutes an AI system include both explicit objectives, as in clearly stated goals that are directly encoded by the developer into the system (e.g. a function to optimise), and implicit objectives, as in preferences that are solely expressed through patterns to apply or mimic (e.g. from the annotations added to training data), including behaviours and other assumptions that can be made when interacting with the environment.

The concept of autonomy in the AI act ranges from none (only manually operated functions) to full automation (no human involvement or intervention during operation), and any level of autonomy beyond zero is constitutive of an AI system. These human-machine interactions include both direct interactions (e.g. manual controls) and indirect ones (e.g. through a supervision system) depending on how much the human is part of the process.

The outputs of the AI system can be of different types depending on the level of human involvement, ranging from estimating values, producing content, suggesting actions, to automatically enacting a choice.

The AI techniques used when building an AI system can involve either or both of:

  • Machine learning approaches, where data is used to learn how to achieve the objectives. This includes deep learning but not only, and it encompasses in practice a large variety of methods and techniques of varied complexity and maturity.

  • Logic- and knowledge-based approaches, where knowledge that is relevant to achieve the objectives is encoded in an explicit (symbolic) way. This includes expert systems but also a broader set of AI techniques such as knowledge representation (e.g. knowledge graphs), inference engines, inductive programming, as well as some search and optimisation methods.

An AI system can be designed with adaptiveness, in the sense of an ability of the system to change while in use by automatically identifying new patterns or relationships, but this is not necessary and many AI systems are not. In such cases the AI system can produce different results when given again the same inputs, but otherwise the outputs are usually repeatable.

Difference from other software systems

While there can be high automation in both cases, AI systems differ from other software systems by the fact that their operation is not solely based on human-defined operations to automatically execute. This does not mean that rule-based systems such as expert systems are not AI systems. In an expert system, humans encode knowledge in the form of processing rules, but then the AI system operates some inference on those: it does not merely follow a stepwise list of instructions, the output is not built incrementally by operations exactly specified by a human.

The European Commission guidelines offer more precise insights by identifying cases with some inference capabilities, but which still fall outside the scope of the AI system definition due to their limited capacity to analyse patterns. This includes:

  • Basic data processing (without any learning, reasoning or modelling), such as running a query on a database, spreadsheet computations, statistical methods to extrapolate estimates and correlations from sampled data, or other techniques for descriptive analysis and visualisation.

  • Systems (even if based on AI techniques, including machine learning) that solely aim at improving the functioning, efficiency or performance of a mathematical optimisation process by speeding it up or running it in an approximate way.

  • Some heuristics-based systems, in the sense of strategies used to find faster, or using less resources, an approximate solution to a problem for which an exact solution could be found programmatically but it is impractical.

  • Simple prediction systems that only apply basic statistics, such as using the mean of known values as an estimate for the desired value.

Boundaries of the AI system

The AI system encompasses both hardware components (the infrastructure for computation) and software components (what runs on that infrastructure):

  • Its software components include models (learned parameters, explicit knowledge representations…) and code (compiled or interpreted) used to run inference with these models or to build them, but also any other code involved when processing the input data into the output (e.g. format conversion, scheduling, API handling, management of computing hardware), as well as the operating system running that code.

  • Its hardware components include processing units (CPU, GPU, TPU, or other forms such as FPGA), memory and storage components, or other elements enabling communication such as networking devices or input/output interfaces.

In the case of an AI system embedded into a physical product, the physical components of the product (beyond the computing infrastructure: actuators, moving pieces, sensors…) are not considered to be within the boundaries of the AI system. However, it does not mean that these are not fully relevant to take into account when developing the AI system or assessing its characteristics, as these external elements can still drive a lot the design or behaviour of the AI system.

See also:

^ The risk-based approach

Risk in EU legislation

Risk is a key concept in EU legislation under the ‘New Legislative Framework’ that regulates the circulation of goods and services within the EU single market. But it does not mean risk in the sense of uncertainty, as in the concepts of ‘risk appetite’ or ‘risk aversion’ that are well-known for instance in finance (a high-risk investment is a very uncertain one that can yield a lot of profit).

In the context of EU legislation, risk refers to the likelihood and severity of a harm: a risk is something that by design is sought to be avoided, and the real key question is how much efforts are necessary to completely eliminate the risk (make that harm impossible or innocuous), and how much of these efforts the regulation requires to perform, as reducing the risk to a given level can be enough.

The risk-based approach in EU legislation means that goods and services are only regulated when they pose certain risks, rather than regulating certain technologies out of principle. The AI Act is built on that risk-based approach: the more risks an AI system creates, the more legal constraints it is subject to (from none, just transparency, technical requirements, up to plain prohibition).

Note: This can appear less intuitive in the case of general-purpose AI models, which are subject to stronger obligations when they present systemic risks, but are still regulated in the general case. However, in this case the primary reason to regulate GPAI models is a value chain one rather than the technology itself: in light of existing practices of integrating or modifying third-party GPAI models, these obligations are meant to ensure sufficient transparency so that downstream providers who integrate the GPAI model in a high-risk AI system have enough information to fulfill their own risk-based obligations.

Being risk-based does not mean that all risks are considered equally. The AI Act is focused specifically on risks to health, safety and fundamental rights. This includes the protection of democracy, the rule of law and the environment. But it does not include for instance the financial risks that the provider can face, or the loss of clients. While the provider can of course seek protection against those risks, it is not made mandatory by the AI Act, and it is not subject to trade-offs with the risks that are actually targeted by the AI Act (e.g. accepting risks to health to reduce a financial risk).

Risk tiers in the AI Act

The AI Act defines four tiers of risks for AI systems. Unacceptable risks (e.g. for social scoring, manipulation) result in prohibitions. AI systems with high risks (e.g. AI in medical devices, employment) are allowed but subject to a series of technical requirements to ensure they are safe enough to use. Limited risks (e.g. impersonation, fake news) are only associated with transparency obligations, to ensure that users are well aware of the risks. Minimal risks are not specifically regulated.

General-Purpose AI models follow their own two-tier classification, where GPAI models in general are mostly subject to transparency obligations while GPAI models with systemic risk are under stronger obligations regarding evaluation, risk assessment and mitigation, reporting and cybersecurity protection.

These tiers are in practice not mutually exclusive. An AI system can be high-risk but also fall under the transparency obligations associated with limited risks, for instance if generating content as part of a medical device. A GPAI model with systemic risk can be integrated into an AI system with minimal risk, or vice versa. In such cases, any obligation whose criteria are met just applies jointly to the AI system or AI model.

The tiers and types of risks are not fixed in time. Given the expected future evolution of AI technologies and usage, the European Commission has been empowered by the European Parliament and the EU Council to amend a number of the criteria distinguishing those tiers and the specific cases that are regulated.

Risk classification and risk assessment

The identification of the risk tier that the AI system or AI model falls in is a distinct process from the estimation of concrete risks. The classification into a tier is primarily based on the type of use cases, which by themselves are usually associated with risks. If the AI system is classified as high-risk or the GPAI model as presenting a systemic risk, then in turn it is subject to a mandatory process for risk assessment, which determines specific risks and mitigation measures to address them (in practice, to reduce them to an acceptable level). But this process is only required once the AI system or GPAI model has already been identified as high-risk or with systemic risk: it is not that process that classifies it as such.

However, in exceptional cases, the categories of risks based on the use case can still appear too coarse and result in inconsistent classification compared to the actual risks. In such cases, the provider can provide their own assessment to justify to the authorities why their AI system needs to be reclassified as not high-risk despite its use case (with specific conditions described in Article 6(3) and (4)), which the authorities can in turn verify and possibly object (Article 80).

The AI Act also contains, as any EU legislation under the New Legislative Framework, a concept of ‘presenting a risk’ (Article 79). This is different from risk classification and from just having identified the risks. It refers in fact specifically to the case where a given risk appears to exist to a degree that is beyond acceptability. This particular case is associated with dedicated logging requirements, certain reporting obligations, and authorities have a special process to investigate such situation and check whether the AI system is effectively compliant or warrants corrective actions or even withdrawal or recall.

Risk and assistive AI

AI systems do not necessarily run in a fully automated mode that directly affects the real world, and many of them are in fact making suggestions that the user can decide to follow or not, or they can provide pure assistance to the user in their own tasks.

It can be tempting to assume that assistive AI systems are not causing any risk and therefore do not fall under the obligations of the AI Act. However, it is actually the other way around. The risk tier is determined based on the category of use case, which causes an obligation of human oversight (Article 14), and making the AI system purely assistive is a good way to offer this human oversight. So this only means that this requirement is fulfilled, not that the requirement does not apply, nor that other requirements for high-risk AI systems do not apply to that purely assistive AI system (e.g. robustness or technical documentation).

See also:

^ Operators in the value and supply chains

Value chain: providers and product manufacturers

The provider is the natural or legal person, or any other body, that develops the AI system or AI model (or gets someone to develop it for them) then places it on the market or puts it into service. This does not imply payment: the provider is still considered as provider even if it does so for free. This also includes in-house development, when an AI system is developed, put into service and used all within the same organisation.

The provider is the one who does this placement on the market or putting into service under its own name or trademark. If another operator (distributor, importer, deployer or any other) puts their name or trademark on an AI system already on the market or in service, then this operator becomes the provider from the legal viewpoint, and the obligations now apply on them rather than on the original provider (except if there is a particular contractual arrangement that allocates responsibilities differently). The original provider remains however due to cooperate with the new provider and communicate any information needed for compliance, except if confidential.

This transfer of responsibilities also occurs in case of a substantial modification to a high-risk AI system (which was already considered high-risk and remains so), as well as when deciding to place on the market, put into service or use a non-high-risk AI system in a way that now becomes high-risk (which formally is a modification of its intended purpose).

When the AI system is a safety component of a product, the product manufacturer is also considered to be the provider of the AI system (and is subject to corresponding obligations) if, under its own name or trademark, it places the AI system on the market together with that product (e.g. AI system embedded in a physical product) or it puts it into service after the product has been placed on the market (e.g. AI system in the cloud enabling a given functionality of a physical product).

An AI system or AI model can be built as a modification, completion or combination of various components, including third-party AI systems and AI models that the new AI system or AI model can integrate or build on. The relationship of the various organisations involved is referred to as the AI value chain. If not properly framed, this relationship can create challenges for the compliance of the final provider, for lack of having access to the needed information or data to offer a sufficiently accurate and comprehensive view on the AI system’s characteristics. In the case of high-risk AI systems (except if the third-party components are from a free and open-source licence), the provider and the third party are required to formalise through written agreement the information that they intend to exchange. In the case of general-purpose AI models, the information to communicate to a downstream provider is directly specified by the AI Act.

Deployers and users

A deployer is a natural or legal person, or any other body, under the authority of which an AI system is used. This means that it assumes responsibility over the decision to deploy the system and over the manner of its actual use. For instance, if the deployer is an employer (e.g. a hospital), the employees using that AI system under the responsibility of their employer (e.g. the medical doctors working at the hospital) are not considered as deployers and the AI Act obligations do not apply on them but on their employer. This implies that deployers are not necessarily end-users themselves.

This role excludes what the AI Act refers to as ‘personal non-professional activities’: a natural person that uses an AI system for themselves, or for leisure, is not considered as a deployer. If they do so in a professional context, and under their own responsibility, then they are deployer.

The provider can also be the deployer (or one of the deployers), if it actually uses the AI system itself. If it puts it into service (e.g. runs an online platform on a server) but the users are external persons not under its responsibility (such as users of a social media platform, not external contractors), then it is solely provider and not deployer.

Supply chain: importers and distributors

While a non-EU provider is expected to appoint an authorised representative within the EU (who has a number of compliance checks to perform), it is not that authorised representative that actually imports and distributes the AI system inside the EU. The importer is the natural or legal person in the EU that places on the market (for the first time) an AI system from a non-EU provider, under that provider’s name.

A distributor is a natural or legal person making the AI system available on the market (not necessarily for the first time, so there can be multiple distributors), who is neither provider nor importer of that AI system.

See also: