When is an AI system high-risk?
5 Conclusion
s. 38–41pp. 38–41 · PDF 42–45
5.1 Answer to the research question#s. 38
Six operative criteria are derived in this thesis from the AI Act's text, systematic context, analogies from GDPR Article 22 and the ECJ's SCHUFA judgment, and information theory, to operationalise the notions of “narrow procedural task” and “preparatory task to an assessment” in Article 6(3)(a) and (d):
1.whether the task involves profiling of natural persons, which categorically precludes the carve-out,
2.whether the task is specific, bounded, and – for Article 6(3)(a) – narrow,
3.whether the operation is syntactic rather than semantic, or – for Article 6(3)(d) – whether it is preparatory without generating evaluative conclusions about the subject matter of the subsequent assessment,
4.whether the output bears on the substance of the decision, assessed through a counterfactual test derived from the SCHUFA “determining role” doctrine,
5.whether the output is transparent and verifiable by the human assessor in the actual conditions of deployment; and
6.whether the output is inherently resistant to uncritical reliance in the absence of Article 14 human-oversight safeguards.
Applied sequentially and cumulatively, these criteria reduce the risk of regulatory arbitrage by providing a structured methodology that prevents substantive decision-support functions from being re-characterised as “procedural” or “preparatory,” while preserving the carve-outs' effet utile for genuinely low-risk administrative and logistical AI functions deployed within Annex III areas.
5.2 Summary of findings and contributions#s. 38
The answer to the research question rests on the interpretive analysis conducted in Chapters 2 through 4. The Article 6(3) carve-out operates as a proportionality “safety valve” on a blanket high-risk presumption within Annex III; the assessment is task-centred rather than system-centred; and the self-assessment model places a premium on the clarity of the substantive criteria. Textual analysis, drawing on information theory, identified the key distinctions: between “narrow” and broad tasks, between “procedural” (syntactic) and substantive (semantic) processing, and between “preparatory” functions and those bearing on the substance of an assessment. The systematic analysis drew on Article 14 on human oversight, the GDPR Article 22 framework on automated individual decision-making, and the ECJ's SCHUFA judgment, which established that the legal characterisation of a task depends on its functional role in the decision-making chain rather than on its formal position in a workflow. The teleological analysis confirmed a two-sided interpretive constraint: the carve-outs must be read strictly enough to prevent substantive decision-support functions from being relabelled as procedural or preparatory, yet broadly enough to preserve their effet utile for genuinely low-risk administrative functions. From this the operationalised criteria were derived.
The illustrative examples summarised in Section 4.3 and developed in full in the Appendix demonstrate the framework's practical utility across Annex III sectors. Applying the criteria to a clinical documentation workflow illustrates a counterintuitive but analytically interesting result: an AI system may lose its Article 6(3)(d) exemption by providing more information — such as confidence scores — that steers the human quality assurance process, even where the underlying classification task itself remains preparatory. The recruitment workflow illustrated the profiling override's decisive filtering function: the moment an AI system begins evaluating personal characteristics of candidates, even through transparent rule-based criteria, the carve-outs are categorically unavailable.
The criteria are intended to serve as a practical interpretive tool for three principal audiences: Providers, who bear the Article 6(4) self-assessment burden and need a structured methodology for documenting their classification; deployers, who under Article 25(1)(c) risk reclassification as providers if they use a system outside its documented intended purpose, and need a basis for critically evaluating the provider’s characterisation; and national market surveillance authorities, who need assessable dimensions for reviewing self-assessments.
5.3 De lege ferenda: improving the legislative framework#s. 39
The analysis reveals weaknesses in the Article 6(3) framework that warrant legislative, Commission, or supervisory clarification. I offer four de lege ferenda reflections:
First, I refer to the analysis in Section 3.1.2, arguing that the relationship between the general “no significant risk of harm” condition and the four enumerated conditions is unclear, but that the better reading is that the four conditions are the sole gateway to the carve-out. Confirming this in Article 6(5) guidance or through textual amendment would improve legal certainty.
Second, the profiling override in the third subparagraph operates as a categorical bar: any AI task that evaluates personal aspects of a natural person triggers the high-risk regime, regardless of whether the evaluation operates to the person’s detriment or to their benefit. This binary treatment may be appropriate for prototypical cases such as credit scoring or recidivism prediction, where the evaluation directly conditions an adverse outcome. It is less obviously appropriate for “Pareto-improving” deployments in which a profiling system clearly supports the person profiled without exposing them to any risk of harm – for example, an employeeonboarding or upskilling tool that profiles a new hire’s existing skills in order to recommend a personalised training pathway, where the recommendation cannot serve as the basis for any decision adverse to the employee and the employee retains the option to disregard it. Capturing such systems within the high-risk regime by virtue of the profiling alone is, in my view, disproportionate. A targeted Article 6(5) clarification, or a textual amendment to the third subparagraph, could permit profiling-based AI to fall within Article 6(3)(d) where the system’s output is used only in the deployer’s stated interest in supporting the person profiled, the output cannot be used as the basis for any adverse decision concerning that person, and the person retains a meaningful right to opt out. With such conditions I believe the cost — making it somewhat harder to police covert evaluative scoring of natural persons — is worth paying to remove the disincentive for genuinely beneficial profiling-based applications in Annex III sectors.
Third, the framework yields a paradoxical result in cases involving confidence reporting on otherwise preparatory tasks, as illustrated by the clinical documentation case study summarised in Section 4.3 and detailed in the Appendix. A system performing speaker labelling or statement labelling without confidence scores passes the derogation criteria, while a system returning the same outputs with confidence scores included does not, because the confidence scores risk steering the human quality assurance. It is somewhat counterintuitive that a system can change classification from “high-risk” to “not-highrisk” simply by offering less information relevant to the subsequent human review. In my view, the result is nevertheless defensible. Article 6(3) is concerned not with the volume of information an AI system produces, but with whether the form of that information meaningfully shifts cognitive effort from the human assessor to the system. A bare classification leaves the assessor in the position of judging whether to trust each label on its own evidentiary footing; a confidence-scored classification invites the assessor to partially delegate that judgment to the system. The Article 14 design requirements are calibrated to the latter case, and so is the high-risk regime as a whole. The open de lege ferenda question, in my view, is whether providers should nevertheless be incentivised to provide such transparency information – for example, through a graduated regime that admits transparent confidence reporting under stricter human-oversight conditions.
Fourth, the self-assessment model of Article 6(4) places the classification burden on the provider, but the analysis in Chapters 3 and 4 has shown that several of the operative criteria – particularly the transparency, verifiability, and automation bias dimensions – depend heavily on the conditions of deployment, on which the provider can only make assumptions. The Act's current architecture addresses this partly through the deployer's obligations under Article 26 and the reclassification mechanism of Article 25(1)(c), but it does not require the provider to specify the deployment conditions under which the Article 6(3) assessment was made beyond the intended purpose “specific context and conditions of use” of Article 6(4) read in conjunction with Article 49(2), Article 71, Annex VIII A(5), and Article 3(12). I submit that — as part of the Article 6(3) and (4) self-assessment and registration — there should be an explicit requirement on providers documenting context and conditions of use to specify in detail the assumed roles and capabilities of the humans in/on the loop, the availability of verification mechanisms, and the organisational context. This would make the contextual dependencies of the classification transparent and enable deployers and supervisory authorities to assess whether the provider's assumptions hold in their specific context.
5.4 Residual uncertainties and directions for future research#s. 41
The boundary between syntactic and semantic processing, while analytically useful, will require further calibration for natural language processing tasks that inherently involve engagement with meaning. The Recital (53) examples of preparatory tasks – text processing, translation, indexing – demonstrate that some degree of semantic engagement is compatible with the Article 6(3)(d) derogation, but the threshold beyond which semantic processing becomes evaluative remains underspecified. The Commission’s forthcoming guidelines under Article 6(5) could usefully address this question by providing a graduated set of examples illustrating where the boundary lies for common language-processing tasks such as summarisation, entity extraction, sentiment classification, topic classification, and relevance ranking.
The risk of mission creep in general-purpose-capable AI systems — where a system assessed as performing a procedural or preparatory task is gradually repurposed for more substantive functions — warrants attention. The Article 6(3) assessment is made at a point in time, anchored to the provider’s documented intended purpose. But the capabilities of modern AI systems, particularly those based on LLMs, often extend well beyond any single documented purpose, and evolve quickly. The existing mechanism of Article 25(1)(c), which reclassifies deployers as providers when they use a system beyond its intended purpose, provides a formal safeguard, but its practical effectiveness depends on the willingness and capacity of supervisory authorities to monitor actual deployment practices.
The framework proposed here has been finalised in early May 2026. It is developed on the basis of doctrinal analysis and information theory and tested against illustrative cases. It has not been subjected to empirical validation against a comprehensive taxonomy of real-world AI deployments across all Annex III areas. As enforcement practice develops and the Commission’s forthcoming Article 6(5) guidelines are adopted, the criteria may need to be refined, supplemented, or reconsidered. The framework is offered as an initial contribution to a discourse that will continue to evolve through the interaction of academic analysis, regulatory guidance, supervisory practice, and — in time — the jurisprudence of the Court of Justice.