When is an AI system high-risk?
4 Operationalising the criteria: a framework for applying Article 6(3)(a) and (d)
s. 29–36pp. 29–36 · PDF 33–40
4.1 From interpretation to operationalisation#s. 29
The following framework is derived from the legal analysis in Chapters 2 and 3 and is offered as a doctrinal contribution to the applied interpretation of Article 6(3)(a) and (d). The criteria are designed to be applied sequentially: each criterion must be satisfied before proceeding to the next. A negative answer at any stage terminates the assessment with the conclusion that the carve-out does not apply. This sequential structure reflects the principle of strict interpretation of derogations: each criterion functions as a necessary condition, and only the cumulative satisfaction of all criteria is sufficient to support the conclusion that the AI system’s task falls within Article 6(3)(a) or (d).
4.2 The proposed criteria#s. 29
4.2.1 Criterion 1: The profiling threshold – does the task involve profiling of natural persons?#s. 29
The assessment starts with determining whether the AI system performs profiling of natural persons within the meaning of the GDPR Article 4(4). As discussed in Section 3.5, the profiling override is absolute: where it applies, the AI system is always high-risk, regardless of any other consideration. The criterion gives a preliminary gate: If the AI system performs profiling, the Article 6(3) carve-outs are categorically unavailable.
Beginning the assessment with this criterion is practical. A significant number of AI systems deployed in Annex III areas — creditworthiness assessment, insurance risk scoring, predictive policing, recidivism prediction, employee performance monitoring — involve profiling tasks by their very nature. For these tasks, any further analysis of the “procedural” or “preparatory” character is superfluous. At the same time, the profiling override applies at the level of the task, not of the domain: Other AI systems within the same workflow — for example, those performing data deduplication, document classification, or speech-to-text transcription — remain eligible for the carve-out where the substantive criteria below are satisfied. The profiling threshold filters out many common assessment-bearing tasks at the outset, concentrating the remaining analysis on the auxiliary tasks that do not involve the evaluative assessment of personal aspects (but might still be high risk through other mechanisms, discussed in the following criteria).
The criterion also has a methodological function. As observed in Section 3.5, the EDPB’s interpretation of profiling emphasises the evaluative element: processing that merely handles personal data without evaluating the person does not constitute profiling. This distinction maps directly onto the boundary between syntactic and semantic processing discussed in Section 3.2.1. An AI system that processes personal data for purely formal purposes – verifying the format of a national identity number, detecting duplicate entries in a database of applicants, or converting a handwritten application form into structured digital text – handles personal data but does not evaluate the person. Such systems pass this first criterion and proceed to the substantive assessment under the remaining criteria.
4.2.2 Criterion 2: Task delimitation – is the task specific, defined, and bounded?#s. 30
The second criterion follows from the textual analysis of “task” in Section 3.2.1 and the “narrow” qualifier applicable to condition (a). To qualify under either Article 6(3)(a) or (d), the function performed by the AI system must constitute a specific, delimited, and bounded piece of work within a broader process. This requirement has three sub-dimensions.
First, the task must be identifiable as a discrete function with clear inputs and outputs. It must be possible to describe what the AI system does in concrete operational terms: what data it takes in, what operations it performs, and what output it produces. An AI system whose function is only described in vague or open-ended terms — “assisting with recruitment decisions” or “supporting insurance assessments” — fails this requirement, because the description does not identify a delimited task but rather a broad functional domain.
Second, the task must have definable boundary conditions: a clear beginning and end within the process in which it is embedded. The function should not extend across multiple stages of the decision-making workflow. An AI system that both collects and organises evidence and then prioritises or ranks it for the decision-maker is performing two functionally distinct tasks, one of which may be preparatory while the other may be substantive. The provider cannot bundle these into a single “task” description and claim the benefit of the carve-out for the composite.
Third, for Article 6(3)(a) specifically, the task must be “narrow”: it must operate at the limited end of the scope spectrum. An AI system performing a single, well-defined data transformation (extracting dates from documents) satisfies the narrowness requirement. An AI system performing a range of data transformation tasks across a broader workflow, even if each individual task is procedural, may not satisfy the narrowness criterion where its combined function spans a significant portion of the information processing chain. The narrowness qualifier, however, applies only to Article 6(3)(a). Article 6(3)(d) speaks of a “preparatory task” without that qualifier, so a preparatory task may be broader in scope than a narrow procedural task, provided the other criteria are met.
4.2.3 Criterion 3: The nature of the operation – syntactic or semantic?#s. 31
The third criterion draws on a central distinction developed in the textual analysis of Article 6(3)(a) and (d): the distinction between syntactic and semantic processing. A task is syntactic where it operates on the form, structure, or format of information without engaging with its evaluative content. A task is semantic where it operates on the meaning, significance, or evaluative implications of information in relation to the decision to be made. The criterion applies to both Article 6(3)(a) and Article 6(3)(d), but with different stringency: Article 6(3)(a) requires the task to be strictly syntactic, while Article 6(3)(d) tolerates a degree of semantic processing provided it does not generate evaluative conclusions about the subject matter of the subsequent assessment. The two cases are addressed in turn below.
For Article 6(3)(a), this criterion is decisive: only tasks that are syntactic in character qualify as “procedural.” The Recital (53) examples – transforming unstructured data into structured data, classifying documents into formal categories, detecting duplicates – are all syntactic operations whose correctness is, in principle, objectively verifiable by reference to formal criteria. A system that categorises incoming insurance claims by document type (medical certificate, police report, invoice) performs a syntactic operation; a system that categorises the same claims by estimated severity or likely outcome performs a semantic one.
For Article 6(3)(d), the syntactic/semantic distinction operates differently. As discussed in Section 3.3.2, a preparatory task may involve a degree of semantic processing – translation, transcription, indexing – that goes beyond purely formal data manipulation. The Recital (53) examples of preparatory tasks include “text and speech processing” and “translation of initial documents,” which inherently involve engagement with the meaning of language. The decisive question under Article 6(3)(d) is therefore not whether the operation is strictly syntactic, but whether it generates evaluative conclusions about the subject matter of the subsequent assessment. A translation makes information accessible in another language; it does not evaluate its significance for the decision. A transcription converts speech to text; it does not assess the credibility or relevance of what was said. An indexing system creates retrievable metadata; it does not determine which information matters.
This yields a more nuanced formulation of Criterion 3 for the two conditions: Under Article 6(3)(a), the task must be strictly syntactic – limited to operations on the form, structure, or format of data. The system must not produce outputs that require the exercise of evaluative judgment to generate, as opposed to the application of formal, rule-based, or deterministic criteria. Under Article 6(3)(d), the task may involve a degree of evaluative semantic processing, but the output must not contain evaluative conclusions about – or assign evaluative weight to information in relation to – the subject matter of the subsequent assessment. This is closely tied to Criterion 4 below. In practice the Article 6(3)(d) formulation will be the most common application of Criterion 3, since all preceding procedural tasks are also by nature preparatory.
4.2.4 Criterion 4: Functional role – does the output bear on the substance of an assessment or decision?#s. 32
Even where a task is delimited and operates within the syntactic or permissible semantic range, the Article 6(3) carve-out requires that it does not bear on the substance of the relevant decision or assessment. This criterion operationalises the SCHUFA doctrine discussed in Chapter 3: the legal characterisation of a task depends not on its formal position in a workflow, but on its functional role in relation to the decision affecting the natural person.
The operative question is: Could a reasonable decision-maker’s conclusion be different depending on how the AI system performs its task, compared to another (differently designed) AI system, or a human assistant? If the answer is yes — if choices made through the AI system’s design, inference model, and output presentation shapes, narrows, or pre-structures the informational basis on which a substantive assessment or decision is made in a way that could alter the outcome — then the AI system’s task bears on the substance of the decision and the carve-out does not apply. If the answer is no — if the AI system’s task is such that performing it with a different system, or having a human perform the task, would not change the substantive information available to the decision-maker (barring outright system malfunctions and human error) — then the task does not bear on the substance of the decision.
This criterion can be further operationalised through a counterfactual test: Would the decision-maker, in a plausible scenario, reach a different substantive conclusion if the AI system was unavailable, and the decision maker had to perform the procedural or preparatory task themselves? For instance, the task of sorting job applications by the applicants’ alma mater does not bear on the substance of a hiring decision: the same applications are before the assessor regardless of their ordering, and no substantive information is added or subtracted. Sorting applications by “relevance” to the job description, by contrast, structures the assessor’s cognitive field in a way that could affect which candidates receive closer attention, bearing directly on the substance of the decision.
As indicated above, what amounts to a “different substantive conclusion” should not be extended to include errors from system malfunctions (or human error) in the otherwise purely procedural or preparatory function itself. The effet utile principle discussed in Section 3.2.3 requires that the carve-out not be deprived of its practical effect. Every AI system is capable of error, and a translation error, a transcription mistake, or even a mis-parsed date could in principle lead a decision-maker to a different conclusion. If the counterfactual test were applied so stringently as to capture every possible error in a procedural or preparatory function, the carve-out would be rendered empty. The question must be to the designed functional role of the output, not to the consequences of malfunction.
4.2.5 Criterion 5: Transparency – is the output transparent and verifiable?#s. 33
The fifth criterion follows from the systematic relationship between Article 6(3) and Article 14 on human oversight, as analysed in Sections 3.2.2 and 3.3.2.2. For a task to be genuinely procedural or preparatory, the output must be transparent and verifiable by the person who performs the subsequent assessment or decision.
Transparency here means that the nature of the operation performed by the AI system must be comprehensible to the person receiving the output. The assessor should understand what the system has done: that it has translated a document, converted speech to text, or extracted dates from a collection of files. Verifiability means that the recipient of the output must be able, in principle, to check the output against the underlying data. A translated document can be checked against the original by a person with the requisite language proficiency. A date extraction can be verified against the source document. A duplicate detection result can be confirmed by comparing the flagged records.
Where the output is opaque — where the system produces a result that the assessor can observe but cannot meaningfully verify — the conditions for the carve-out are not satisfied. The distinction between ex ante transparency (system-level information) and ex post explanation (decision-specific reasoning) is relevant here. The Article 6(3) carve-out does not require full algorithmic transparency, but it does require that the output be of a kind that a competent assessor can critically evaluate. A list of extracted data fields is verifiable. A probabilistic relevance score, generated by an opaque model, is not – or at least not without the kind of technical infrastructure that Article 14 is designed to mandate.
This criterion also has a contextual dimension. Whether an output is verifiable depends not only on the nature of the output itself but on the capabilities and resources of the person who receives it. As noted in Section 3.3.2.2, translating an asylum application from Urdu to Norwegian is verifiable only if the assessor has access to proficiency in the original language. A system that is deployed in a context where practical verification by the assessor is impossible is, from a functional perspective, not producing a transparently verifiable output – even if the output is, in abstract, of a kind that could be verified. The verifiability assessment must be made with reference to the actual conditions of deployment, not to an idealised scenario.
4.2.6 Criterion 6: Automation bias resilience – is the output inherently resistant to uncritical reliance?#s. 34
The final criterion addresses the risk of automation bias; a concern identified in Sections 3.2.2 and 3.3.2 as central to the systematic interpretation of both conditions (a) and (d). AI systems exempted under Article 6(3) are relieved of the very human oversight design requirements that are intended to counteract automation bias. The exemption is justified only where the nature of the AI system’s task and output is such that the risk of uncritical reliance is inherently limited.
This criterion asks whether the output of the AI system is of a kind that, by its nature, invites independent assessment rather than deference. Factual, discrete outputs – a list of extracted dates, a translated document, a set of identified duplicate records – are inherently resistant to automation bias because they present information without suggesting a course of action. They do not tell the decision-maker what to think about the information; they merely make the information available in a usable form. By contrast, outputs that aggregate, evaluate, or assign probabilities — an executive summary emphasising “key findings,” a relevance ranking, a risk score — are inherently susceptible to automation bias because they compress complex information into a cognitively convenient form that the decision-maker is incentivised to adopt rather than scrutinise.
The AI Act’s treatment of automation bias has been criticised for requiring awareness of the risk without adequately aligning provider and deployer obligations with the cognitive, designrelated and contextual mechanisms through which automation bias arises, despite Article 14’s provider-side human-oversight design duties and Article 26’s deployer-side competence and training obligations. This concern reinforces the importance of Criterion 6 for the Article 6(3) assessment: if even the Article 14 and 26 safeguards are not to apply, it follows systematically that the nature of the task and its output must be such that the risk of cognitive reliance is inherently very low. The assessment must consider both the formal characteristics of the output and the cognitive dynamics of the context in which it is received. An output that appears “neutral” in isolation may become decision-determinative when presented to an overworked assessor processing a high volume of cases under time pressure.
4.3 Application to illustrative cases#s. 35
The framework is tested in three illustrative case studies detailed in the Appendix, each drawn from a distinct Annex III sector. The summaries below highlight what each case contributes to the analysis; the criterion-by-criterion application is set out in full in the Appendix.
The first case examines an AI system used to translate asylum applicants’ written statements into the language of the receiving state, the translation feeding into the eligibility assessment by the case officer. The case illustrates the contextual sensitivity of Criterion 5: the same AI system may qualify for the Article 6(3)(d) carve-out in one deployment context and not in another, depending on whether the case officer has practical access to native-language proficiency to verify the translation. Where verification is illusory, a translation that is in abstract verifiable becomes functionally determinative, and the carve-out does not apply.
The second case examines a multi-stage recruitment workflow – data extraction, rule-based scoring, soft-criteria evaluation, and final ranking. It illustrates the filtering function of the profiling override: the earliest stage (data extraction) qualifies for the carve-out under both Article 6(3)(a) and (d), while each subsequent stage that introduces evaluative engagement with candidates’ personal aspects – even through transparent rule-based criteria – is foreclosed by the third-subparagraph override. The result is a characteristic gradient: AI can support the administrative preconditions of recruitment without engaging the high-risk regime, but the moment it begins evaluating the candidate, the carve-out is categorically unavailable.
The third case examines a clinical documentation workflow comprising audio transcription, speaker classification, statement classification, removal of non-relevant content, and confidence scoring of classifications. It demonstrates, on the one hand, that AI systems can perform highly semantic operations and still qualify for the Article 6(3)(d) carve-out where the operation is undeniably preparatory and presents no significant risk of steering the subsequent assessment. On the other hand, it produces a paradoxical result discussed in Section 5.3 below: adding confidence scores to an otherwise carve-out-eligible classification task takes the system out of the carve-out, because the confidence scoring directs the human assessor’s attention in a manner that bears on the quality and thoroughness of the medical review.
4.4 On the interrelation of criteria 3 to 6#s. 36
Criteria 3 to 6 are interrelated aspects of a single underlying inquiry: whether the AI system’s task remains functionally separate from the substantive assessment or decision. They are not collapsible into a single test that covers both 6(3)(a) and (d) pathways, however, because the boundary between procedural/preparatory and substantive is itself multi-dimensional.
Criterion 3 asks what kind of operation the system performs: does it process the form of information (syntactic) or its evaluative content (semantic)? A system may be syntactic in its operation – producing the output through formal, rule-based parsing – yet substantive in its functional role, where that output has strong impact on an outcome: e.g. a date-extraction algorithm whose parsed value is solely relied upon to decide whether a statutory time limit has been met. Such a system passes Criterion 3 but fails Criterion 4. Conversely, a system may perform semantic processing, translation, transcription, whose output does not bear on the substance of the subsequent assessment because it merely makes information accessible without assigning evaluative weight. Such a system may fail Criterion 3's strict syntactic test under Article 6(3)(a) yet still qualify under Article 6(3)(d) if Criterion 4 is satisfied. Finally, a non-preparatory procedural system (e.g. generating documentation after a decision has been made) may fail the Criterion 3 syntactic test while passing Criterion 4.
Criterion 4 asks what functional role the output plays: does it contribute to the substance of the decision? This is a question about the output's relationship to the decision-making chain, not about the nature of the operation that produced it. The SCHUFA-derived counterfactual test operationalises this inquiry. A system that performs a syntactic operation may still fail Criterion 4 if its output, despite being formally procedural, plays a “determining role” in the subsequent assessment. Criterion 4 thus catches cases that Criterion 3 alone would miss.
Criterion 5 asks whether the output is transparent and verifiable, whether the human assessor can check the AI's work. This is analytically distinct from both the nature of the operation (Criterion 3) and the functional role of the output (Criterion 4). A system may perform a semantic but non-evaluative operation (passing Criterion 3 for Article 6(3)(d)) whose output does not bear on the substance of the decision (passing Criterion 4) but which is nevertheless opaque, e.g. a search system that uses a proprietary relevance algorithm that the assessor cannot interrogate to retrieve and translate news articles for anti-money-laundering know-your-customer research. Such a system may fail Criterion 5 even though it passes Criteria 3 and 4.
Criterion 6 adds the cognitive dimension. Even transparent outputs with no formal bearing on the substance of the decision may invite complacent deference if presented as confidence scores, rankings, or colour-coded priorities. The clinical documentation confidence-score example summarised in Section 4.3 and developed in full in the Appendix illustrates this dynamic.
In short, Criterion 3 asks what the system does; Criterion 4 asks what the output means for the decision; Criterion 5 asks whether the human can check; and Criterion 6 asks whether the human will check. Each captures a dimension that the others, on their own, would miss.