› doceo › document › I… · Web viewWhereas there are serious concerns that the current Union legal framework, including the consumer law acquis, data protection legislation,

European Parliament2019-2024

Committee on the Internal Market and Consumer Protection

2020/2012(INL)

19.5.2020

AMENDMENTS1 - 137Draft opinionAlexandra Geese(PE648.496v01-00)

Framework of ethical aspects of artificial intelligence, robotics and related technologies(2020/2012(INL))

AM\1205570EN.docx PE652.393v01-00

EN United in diversity EN

AM_Com_NonLegOpinion

PE652.393v01-00 2/67 AM\1205570EN.docx

EN

Amendment 1Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-MarquesDraft opinionCitation 1 a (new)

Draft opinion Amendment

- Whereas there are serious concerns that the current Union legal framework, including the consumer law acquis, data protection legislation, product safety and market surveillance legislation, is not fit for purpose to effectively tackle the risks created by artificial intelligence, robotics and related technologies and does not provide a high level of consumer protection as required in Article 38 of the Charter of Fundamental Rights of the European Union;

Or. en

Amendment 2Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-MarquesDraft opinionCitation 1 b (new)


- Whereas ethical guidance, such as the principles adopted by the High-Level Expert Group on Artificial Intelligence, provides a good starting point but is not enough to ensure that businesses act fairly and guarantee effective consumer protection;

Or. en

Amendment 3Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-Marques

AM\1205570EN.docx 3/67 PE652.393v01-00

EN

Draft opinionCitation 1 c (new)


- Whereas artificial intelligence, robotics and related technologies may put consumers at risk of being manipulated and subject to discriminatory treatment and arbitrary, intransparent decisions, thereby contributing to increasing asymmetries of power between businesses and consumers, placing consumers at an even more vulnerable position;

Or. en

Amendment 4Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-MarquesDraft opinionCitation 1 d (new)


- Whereas transparent, accountable and inclusive processes to draft, enact and evaluate policies and legislation applicable to the design, development and deployment of algorithmic systems are of utmost importance to ensure that all individuals that are directly impacted have a meaningful say in whether and how these systems are used, and in whose interest;

Or. en

Amendment 5Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Evelyne Gebhardt, Marc Angel, Adriana Maldonado López, Maria Grapini, Sándor Rónai, Brando Benifei, Andreas SchiederDraft opinionParagraph 1


1. Underlines the importance of an EU 1. Underlines the importance of an EU

PE652.393v01-00 4/67 AM\1205570EN.docx

EN

regulatory framework being applicable where consumers within the Union are users of or subject to an algorithmic system, irrespective of the place of establishment of the entities that develop, sell or employ the system;

regulatory framework being applicable where consumers within the Union are users of or subject to an algorithmic system, irrespective of the place of establishment of the entities that develop, sell or employ the system; furthermore, believes that the rules set out should apply across the value chain, namely development, deployment and use of the relevant technologies and their components and for all developers, and should guarantee the highest level of consumer protection; proposes that these rules take into account the lessons drawn from the adaptation to Regulation (EU) 2016/6791a (GDPR), considered a global benchmark; considers the designation of a liable entity in the Union (such as authorised representative) important for its enforcement;

__________________1a Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (OJ L 119, 4.5.2016, p. 1).

Or. en

Amendment 6Maria da Graça Carvalho, Edina Tóth, Andrey Kovatchev, Marion Walsmann, Pascal ArimontDraft opinionParagraph 1


1. Underlines the importance of an EU regulatory framework being applicable where consumers within the Union are users of or subject to an algorithmic system, irrespective of the place of establishment of the entities that develop, sell or employ the system;

1. Underlines the importance of an EU regulatory framework of ethical aspects of artificial intelligence , robotics and related technologies being applicable where consumers within the Union are users of or subject to an algorithmic system, irrespective of the place of

AM\1205570EN.docx 5/67 PE652.393v01-00

EN

establishment of the entities that develop, sell or employ the system in order to bring legal certainty to business and citizens alike;

Or. en

Amendment 7Svenja Hahn, Claudia Gamon, Dita Charanzová, Jordi Cañas, Sandro Gozi, Stéphanie Yon-CourtinDraft opinionParagraph 1



1. Underlines the importance of an EU regulatory framework of ethical aspects being applicable where consumers within the Union are users of or subject to an algorithmic system, irrespective of the place of establishment of the entities that develop, sell or employ the system;

Or. en

Amendment 8Dita Charanzová, Svenja HahnDraft opinionParagraph 1



1. Underlines the importance of an EU regulatory framework being applicable where consumers within the Union are directed towards or targeted by an algorithmic system, irrespective of the place of establishment of the entities that develop, sell or employ the system;

Or. en

Amendment 9Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Evelyne Gebhardt, Marc Angel, Alex Agius Saliba, Adriana Maldonado López, Maria Grapini, Sándor Rónai, Brando Benifei, Andreas SchiederDraft opinionParagraph 2

PE652.393v01-00 6/67 AM\1205570EN.docx

EN


2. Notes that the framework should apply to algorithmic systems, including the fields of artificial intelligence, machine learning, deep learning, automated decision making processes and robotics;

2. Notes that the framework should apply to algorithmic systems, including the fields of artificial intelligence (AI), internet of things (IoT), machine learning, deep learning, automated decision making processes and robotics; further notes that referral systems should be developed to help explain those systems to consumers whenever they present complexity or constitute decisions that impact their lives significantly;

Or. en




2. Notes that the framework should apply to algorithmic systems, including the fields of artificial intelligence, machine learning, deep learning, automated and assisted decision making processes and robotics;

Or. en

Amendment 11Róża Thun und Hohenstein, Maria da Graça CarvalhoDraft opinionParagraph 2



2. Notes that the regulatory framework should apply to algorithmic systems, including the fields of artificial intelligence, machine learning, deep learning, automated decision making processes and robotics;

AM\1205570EN.docx 7/67 PE652.393v01-00

EN

Or. en

Amendment 12Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Svenja HahnDraft opinionParagraph 2



2. Notes that the framework should apply to algorithmic systems, including the fields of artificial intelligence, machine learning, rule-based systems, automated decision making processes and robotics;

Or. en

Amendment 13Svenja Hahn, Claudia Gamon, Dita Charanzová, Sandro Gozi, Stéphanie Yon-CourtinDraft opinionParagraph 2 a (new)


2a. Believes that the Commission should complete a full review of the existing legislation in order to identify legislative gaps; underlines in that regard the extensive legislation already in force that guarantees for instance that products and services placed on the Union market are safe and do not harm people, respect their privacy and follow stringent environmental rules; calls on the Commission to refrain from adopting a legislative act that would double, overlap or contradict those sector-specific legislations;

Or. en

PE652.393v01-00 8/67 AM\1205570EN.docx

EN

Amendment 14Maria da Graça Carvalho, Róża Thun und Hohenstein, Edina Tóth, Marion WalsmannDraft opinionParagraph 2 a (new)


2a. Stresses that an EU regulatory framework of AI shall have a human-centric approach and lead to development of systems which incorporate European ethical values by-design; considers that an EU regulatory framework that focuses on European values would be an added value providing Europe with a unique competitive advantage and make a significant contribution to the well-being and prosperity of European citizens and businesses, and boost our internal market;

Or. en

Amendment 15Svenja Hahn, Claudia Gamon, Dita Charanzová, Sandro Gozi, Stéphanie Yon-CourtinDraft opinionParagraph 2 b (new)


2b. Points out that the legislative framework introduced by Decision No 768/2008/EC1a provides for a harmonised list of obligations for producers, importers and distributors, encourages the use of standards and foresees several levels of control depending on the dangerousness of the product; considers that this framework should also apply to AI imbedded products;

__________________1a Decision No 768/2008/EC of the European Parliament and of the Council of 9 July 2008 on a common framework for the marketing of products, and repealing Council Decision 93/465/EEC (OJ L 218, 13.8.2008, p. 82).

AM\1205570EN.docx 9/67 PE652.393v01-00

EN

Or. en

Amendment 16Maria da Graça Carvalho, Edina TóthDraft opinionParagraph 2 b (new)


2b. Underlines that an ethical framework of AI is an added value to promote innovation on the market;

Or. en

Amendment 17Svenja Hahn, Claudia Gamon, Dita Charanzová, Sandro Gozi, Stéphanie Yon-CourtinDraft opinionParagraph 2 c (new)


2c. Considers that for the future legislation to apply, legal obligations need to be very precise and avoid to refer to general principles in order to ensure that they are implementable by economic operators;

Or. en

Amendment 18Svenja Hahn, Claudia Gamon, Dita CharanzováDraft opinionParagraph 2 d (new)


2d. Considers that mandatory rules as regards the regulatory framework of ethical aspects should be limited to practices that would undoubtedly undermine fundamental rights and freedoms;

Or. en

Amendment 19Svenja Hahn, Claudia Gamon, Dita Charanzová

PE652.393v01-00 10/67 AM\1205570EN.docx

EN

Draft opinionParagraph 3


3. Stresses that any future regulation should follow a differentiated risk-based approach, based on the potential harm for the individual as well as for society at large, taking into account the specific use context of the algorithmic system; legal obligations should gradually increase with the identified risk level; in the lowest risk category there should be no additional legal obligations; algorithmic systems that may harm an individual, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules;

3. Stresses that any future regulation of ethical aspects should follow a differentiated risk-based approach, based on the potential harm for the individual as well as for society at large, taking into account the specific use context of the algorithmic system; legal obligations should gradually increase with the identified risk level; in the lowest risk categories there should be no additional legal obligations; the risk-based approach should follow clear and transparent rules;

Or. en

Amendment 20Maria da Graça Carvalho, Edina Tóth, Andrey KovatchevDraft opinionParagraph 3



3. Stresses that any future regulation should follow a differentiated risk-based approach, with clear criteria and indicators, followed by an impartial and regulated assessment based on the potential harm for the individual as well as for society at large, taking into account the specific use context of the algorithmic system; legal obligations should gradually increase with the identified risk level; in the lowest risk category there should be no additional legal obligations; this risk-based approach should follow by clear and transparent rules;

AM\1205570EN.docx 11/67 PE652.393v01-00

EN

Or. en

Amendment 21Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Marc Angel, Adriana Maldonado López, Brando Benifei, Andreas SchiederDraft opinionParagraph 3



3. Stresses that any future regulation should follow a differentiated risk-based approach, based on the potential harm or breaches of rights for the individual as well as for society at large, taking into account the specific use context of the algorithmic system; legal obligations and certification requirements should gradually increase with the identified risk level; in the lowest risk category there should be no additional legal obligations and certification or labelling systems should be voluntary; algorithmic systems that may harm an individual or cause potential breaches of an individual’s rights, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules, providing enough legal certainty whilst being future-proof;

Or. en

Amendment 22Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna CavazziniDraft opinionParagraph 3


3. Stresses that any future regulation should follow a differentiated risk-based approach, based on the potential harm for the individual as well as for society at large, taking into account the specific use

3. Stresses that any future regulation should follow a differentiated risk-based approach, based on the potential harm for the individual as well as for society at large, taking into account the specific use

PE652.393v01-00 12/67 AM\1205570EN.docx

EN

context of the algorithmic system; legal obligations should gradually increase with the identified risk level; in the lowest risk category there should be no additional legal obligations; algorithmic systems that may harm an individual, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules;

context of the algorithmic system; legal obligations should gradually increase with the identified risk level; in the lowest risk category there should be no additional legal obligations, whereas applications in the highest risk category should be deemed illegal; algorithmic systems that may harm an individual, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules; the risk assessment of a specific system must be subject to regular re-evaluation;

Or. en

Amendment 23Evžen TošenovskýDraft opinionParagraph 3



3. Stresses that any possible future regulation should enable development and deployment of secure and trustworthy algorithmic systems and that it should follow a differentiated risk-based approach, based on the potential harm for the individual as well as for society at large, taking into account the specific use context of the algorithmic system; legal obligations should gradually increase with the identified basic, substantial or high risk level; in the lowest risk category there should be no additional legal obligations; algorithmic systems that may harm an individual, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules;

Or. en

AM\1205570EN.docx 13/67 PE652.393v01-00

EN




3. Stresses that any future regulation should follow a differentiated risk-based approach, based on the potential harm for the individual as well as for society at large, taking into account the specific use context of the algorithmic system; legal obligations should only increase in accordance with the proven risk level; in the lowest risk category there should be no additional legal obligations; algorithmic systems that may harm an individual, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules;

Or. en


PE652.393v01-00 14/67 AM\1205570EN.docx

EN



3. Stresses that any future regulation should follow a differentiated risk-based approach, based on the potential harm for the individual as well as for the whole society, taking into account the specific use context of the algorithmic system; legal obligations should gradually increase with the identified risk level; in the lowest risk category there should be no additional legal obligations; algorithmic systems that may harm an individual, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules;

Or. en

Amendment 26Evelyne Gebhardt, Marc Angel, Monika Beňová, Alex Agius Saliba, Andreas Schieder, Maria Grapini, Sylvie GuillaumeDraft opinionParagraph 3



3. Stresses that any future regulation should follow a differentiated risk-based approach, based on the potential harm for the individual as well as for society at large, taking into account the specific use context of the algorithmic system; legal obligations should gradually increase with the identified risk level; in the lowest risk category there should be a labelling obligation; algorithmic systems that may harm an individual, impact an individual’s access to resources, or concern their participation in society shall not be deemed to be in the lowest risk category; this risk-based approach should follow clear and transparent rules;

Or. en

AM\1205570EN.docx 15/67 PE652.393v01-00

EN

Amendment 27Evžen TošenovskýDraft opinionParagraph 3 a (new)


3a. Believes that prior to tabling possible new legislative proposals, the Commission should scrutinize application of the legislation already in place and its enforcement, as well as self-regulatory measures; stresses that any legislative proposals related to algorithmic systems should not hamper the emergence of high-tech ‘unicorns’, start-ups and SMEs in Europe or prevent European companies from adopting and implementing AI or other solutions and that such proposals should take into consideration the context of the post COVID-19 crisis period; reiterates that the main objective should be to unleash the potential of algorithmic systems throughout the Union and to increase competitiveness of European companies on a global scale;

Or. en

Amendment 28Jordi CañasDraft opinionParagraph 3 a (new)


3a. Stresses that a framework of ethical aspects should include provisions serving as a reference for engineers, developers, companies and other operators, with the ultimate aim of promoting responsible and ethical deployment, selling and use of AI, robotics and related technologies, as well as for the development of technical standards and certification procedures in the Union;

PE652.393v01-00 16/67 AM\1205570EN.docx

EN

Or. en

Amendment 29Clara Ponsatí ObiolsDraft opinionParagraph 3 a (new)


3a. Calls for the framework to prohibit the deployment or use by Member States of remote recognition technologies, such as biometric recognition that automatically identifies individuals, even for the purpose of responding to a national emergency;

Or. en

Amendment 30Evelyne Gebhardt, Maria-Manuel Leitão-Marques, Marc Angel, Monika Beňová, Alex Agius Saliba, Andreas Schieder, Maria Grapini, Sylvie Guillaume, Sándor RónaiDraft opinionParagraph 3 a (new)


3a. Believes that the ethical principles should be the basis for a harmonised European system of risk classification and related legal obligations;

Or. en

Amendment 31Maria-Manuel Leitão-Marques, Christel Schaldemose, Alex Agius Saliba, Adriana Maldonado López, Maria Grapini, Brando Benifei, Andreas SchiederDraft opinionParagraph 4


4. Underlines the importance of an ethical and regulatory framework including in particular provisions on the quality of data sets used in algorithmic systems, especially regarding the representativeness of training data used, on the de-biasing of data sets, as well as on the algorithms

4. Underlines the importance of an ethical and regulatory framework, including in particular provisions on the quality of data sets used in algorithmic systems in relation to the purpose of its use, especially regarding the representativeness of training data used, on

AM\1205570EN.docx 17/67 PE652.393v01-00

EN

themselves, and on data and aggregation standards;

the de-biasing of data sets, as well as on the algorithms themselves, and on data and aggregation standards; stresses that those data sets should be auditable and made available to the competent authorities whenever called upon to ensure their fitness with the previously exposed principles;

Or. en



4. Underlines the importance of an ethical and regulatory framework including in particular provisions on the quality of data sets used in algorithmic systems, especially regarding the representativeness of training data used, on the de-biasing of data sets, as well as on the algorithms themselves, and on data and aggregation standards;

4. Observes that availability and quality of data is key to address risks which derive from application of the algorithmic systems; underlines the importance of an ethical and regulatory framework including in particular provisions on the quality of data sets used in algorithmic systems, especially regarding the representativeness of training data used, on the de-biasing of data sets, as well as on the algorithms themselves, and on data and aggregation standards;

Or. en




4. Underlines the importance of unlocking large amounts of high quality data to train AI systems; highlights that an ethical regulatory framework should include provisions on the quality of data sets used in algorithmic systems, for instance regarding the representativeness of training data used, on the de-biasing of data sets, as well as on the algorithms

PE652.393v01-00 18/67 AM\1205570EN.docx

EN

themselves, and on data and aggregation standards;

Or. en

Amendment 34Svenja Hahn, Claudia Gamon, Dita Charanzová, Jordi CañasDraft opinionParagraph 4



4. Underlines the importance of a framework of ethical aspects including in particular provisions on the quality of data sets used in algorithmic systems, especially regarding the representativeness of training data used, on the de-biasing of data sets, and on data and aggregation standards;

Or. en

Amendment 35Maria da Graça Carvalho, Edina Tóth, Andrey Kovatchev, Róża Thun und Hohenstein, Pascal ArimontDraft opinionParagraph 4



4. Underlines the importance of an ethical and regulatory framework including in particular provisions on the quality of data sets used in algorithmic systems, especially regarding the high-quality and, where possible, unbiased data sets in order to improve the output of algorithmic systems and boost consumer trust and acceptance;

Or. en

Amendment 36Kateřina Konečná, Anne-Sophie PelletierDraft opinionParagraph 4 a (new)

AM\1205570EN.docx 19/67 PE652.393v01-00

EN


4a. Highlights that user safety, data security, protection of personal data and ethical concerns altogether will determine public acceptance and consequent market penetration of automated systems; highlights that public authorities and private stakeholders will need to provide credible answers to all these concerns as well as prove the environmental, economic, social and safety benefits of AI in order to gain public trust;

Or. en

Amendment 37Jordi CañasDraft opinionParagraph 4 a (new)


4a. Points out that a framework of ethical aspects of artificial intelligence, robotics and other technologies should particularly provide for legally binding measures and standards to prevent practices from private and public actors involved that could undermine fundamental rights and freedoms, in order to provide the highest level of consumer protection and to ensure the development of trustworthy, ethical and technically robust applications in the Single Market;

Or. en

Amendment 38Virginie Joron, Jean-Lin Lacapelle, Alessandra Basso, Marco CampomenosiDraft opinionParagraph 4 a (new)


4a. Stresses that the personal data of European citizens should preferably be

PE652.393v01-00 20/67 AM\1205570EN.docx

EN

processed in Europe.

Or. fr

Amendment 39Kateřina Konečná, Anne-Sophie PelletierDraft opinionParagraph 4 b (new)


4b. Notes that data security and privacy will come along with ethical concerns regarding the definition of the data to collect as well as their ownership, sharing, storage and purpose; notes, additionally, that ethics will play a key role in the definition of the legislative framework regulating the use and management of such data;

Or. en

Amendment 40Kateřina Konečná, Anne-Sophie PelletierDraft opinionParagraph 4 c (new)


4c. Reiterates European principles on the ownership of individuals of their own personal data and explicit, informed consent which is necessary before using personal data as enshrined in the GDPR; points out that consent implies that individuals understand for which purpose their data will be used and that entities using personal data in algorithms have a responsibility for ensuring this understanding;

Or. en

Amendment 41Dita CharanzováDraft opinionParagraph 5

AM\1205570EN.docx 21/67 PE652.393v01-00

EN


5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possible outcome of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected;

5. Believes that consumers have the right to be informed in an understandable, and accessible manner about the existence, process, and reasoning of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected; believes that consumers should also be protected by the right to switch off or limit an AI system using personalisation where possible;

Or. en

Amendment 42Maria da Graça Carvalho, Edina TóthDraft opinionParagraph 5



5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence and reasoning of algorithmic systems, on how to reach a human with decision-making powers, and about how the system’s decisions can be checked, contested and corrected;

Or. en

Amendment 43Alexandra Geeseon behalf of the Greens/EFA GroupRóża Thun und Hohenstein, Kim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-Marques, Svenja HahnDraft opinionParagraph 5


5. Believes that consumers should be 5. Underlines that consumer trust is

PE652.393v01-00 22/67 AM\1205570EN.docx

EN

adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possible outcome of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected;

essential for the development and implementation of artificial intelligence, robotics and related technologies which can carry inherent risks when they are based on opaque algorithms and biased data sets; believes that consumers should be adequately informed in a timely, accurate, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possible outcome of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected;

Or. en

Amendment 44Arba KokalariDraft opinionParagraph 5



5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possible outcome of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected; stresses the importance of proportionality in the development of a transparency framework to avoid unnecessary burdens on SMEs operating in low-risk sectors and to protect intellectual property rights;

Or. en

Amendment 45Maria-Manuel Leitão-Marques, Christel Schaldemose, Marc Angel, Alex Agius Saliba, Adriana Maldonado López, Brando BenifeiDraft opinionParagraph 5

AM\1205570EN.docx 23/67 PE652.393v01-00

EN



5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and intended and reasonably foreseeable unintended outcome of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected; this information should not, however, endanger trade secrets and other competitive advantages;

Or. en

Amendment 46Christel Schaldemose, Leszek Miller, Evelyne Gebhardt, Marc Angel, Alex Agius Saliba, Sándor Rónai, Adriana Maldonado López, Maria Grapini, Andreas Schieder, Sylvie Guillaume, Brando BenifeiDraft opinionParagraph 5



5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possible outcome of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected; recalls that humans must always be able to overrule automated decisions that are final and permanent;

Or. en

PE652.393v01-00 24/67 AM\1205570EN.docx

EN

Amendment 47Evelyne Gebhardt, Maria-Manuel Leitão-Marques, Marc Angel, Monika Beňová, Alex Agius Saliba, Andreas Schieder, Sylvie Guillaume, Sándor RónaiDraft opinionParagraph 5



5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence, the aims or purpose, process, rationale, reasoning and possible outcome and consequences for consumers of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected;

Or. en

Amendment 48Kateřina KonečnáDraft opinionParagraph 5



5. Believes that consumers should be adequately informed, if requested in offline format, in a timely, impartial, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possible outcome of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected;

Or. en

Amendment 49Svenja Hahn, Claudia Gamon, Dita CharanzováDraft opinionParagraph 5

AM\1205570EN.docx 25/67 PE652.393v01-00

EN



5. Believes that consumers should be adequately informed in a timely, impartial, easily-readable, standardised and accessible manner about the existence of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected;

Or. en




5. Believes that consumers should be adequately informed in a timely, accurate, easily-readable, standardised and accessible manner about the existence, process, rationale, reasoning and possible outcome of algorithmic systems, about how to reach a human with decision-making powers, and about how the system’s decisions can be checked, meaningfully contested and corrected;

Or. en

Amendment 51Alexandra Geeseon behalf of the Greens/EFA GroupRóża Thun und Hohenstein, Kim Van Sparrentak, Petra De Sutter, Anna CavazziniDraft opinionParagraph 5 a (new)


5a. Stresses the need to effectively address the challenges created by algorithmic systems and to ensure that consumers are empowered and properly

PE652.393v01-00 26/67 AM\1205570EN.docx

EN

protected; underlines the need to look beyond the traditional principles of information and disclosure on which the consumer acquis has been built, as stronger consumer rights and clear limitations regarding the development and use of algorithmic systems will be necessary to ensure technology contributes to making consumers’ lives better and evolves in a way that respects fundamental and consumer rights and European values;

Or. en



5a. Highlights the need to pay particular attention to situations involving more vulnerable groups such as children, persons with disabilities, elderly people and others that have historically been disadvantaged or are at risk of exclusion, and to situations which are characterised by asymmetries of power or information, such as between employers and workers, or between businesses and consumers;

Or. en

Amendment 53Maria da Graça Carvalho, Edina Tóth, Róża Thun und HohensteinDraft opinionParagraph 5 a (new)


5a. Considers that a value-sensitive design approach is strongly needed to allow a widespread social acceptance of AI for consumers; considers that ethical values of fairness, accuracy, confidentiality and transparency should

AM\1205570EN.docx 27/67 PE652.393v01-00

EN

be the basis of AI which in this context entails that the system’s operations cannot generate unfairly biased outputs;

Or. en



5a. Believes that a European legal framework for a system of risk classification and related legal obligations is required to ensure a uniform protection of European consumers;

Or. en



5b. Believes that explicability is crucial for building and maintaining users’ trust in AI systems; this calls for processes to be transparent, the capabilities and purpose of AI systems openly communicated, and decisions explainable to those directly and indirectly affected; believes that ‘black box’ algorithms which do not provide such information must be required to provide that the system as a whole respects fundamental rights;

Or. en

Amendment 56Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-Marques

PE652.393v01-00 28/67 AM\1205570EN.docx

EN



6. Recalls the importance of ensuring the availability of effective remedies for consumers and calls on the Member States to ensure that accessible, affordable, independent and effective procedures are available to guarantee an impartial review of all claims of violations of consumer rights through the use of algorithmic systems, whether stemming from public or private sector actors;

6. Recalls the importance of ensuring the availability of effective remedies for consumers and calls on the Member States to ensure that accessible, affordable, independent and effective procedures are available to guarantee an impartial review of all claims of violations of consumer rights through the use of algorithmic systems, whether stemming from public or private sector actors; urges that dispute resolution and collective redress mechanisms should be made available both offline and online to consumers, groups and legal entities, who wish to contest the introduction or ongoing use of a system with potential for consumer rights violations, or remedy a violation of rights;

Or. en

Amendment 57Kateřina Konečná, Anne-Sophie PelletierDraft opinionParagraph 6



6. Recalls the importance of ensuring the availability of effective remedies for consumers and calls on the Member States to ensure that accessible, affordable, independent and effective procedures are available to guarantee an impartial review of all claims of violations of consumer rights through the use of algorithmic systems, whether stemming from public or private sector actors; urges Member States to ensure consumer organisations have sufficient funding to assist consumers to exercise their right to remedy;

Or. en

AM\1205570EN.docx 29/67 PE652.393v01-00

EN

Amendment 58Maria da Graça Carvalho, Edina Tóth, Róża Thun und Hohenstein, Andrey Kovatchev, Marion Walsmann, Pascal ArimontDraft opinionParagraph 6



6. Recalls the importance of ensuring the availability of effective remedies for consumers and calls on the Member States and national market surveillance authorities to ensure that accessible, affordable, independent and effective procedures and review structures are available to guarantee an impartial human review of all claims of violations of consumer rights through the use of algorithmic systems, whether stemming from public or private sector actors;

Or. en

Amendment 59Virginie Joron, Jean-Lin Lacapelle, Alessandra Basso, Marco CampomenosiDraft opinionParagraph 7


7. Stresses that where public money contributes to the development or implementation of an algorithmic system, the code, the generated data -as far as it is non-personal- and the trained model should be public by default, to enable transparency and reuse, among other goals, to maximise the achievement of the Single Market, and to avoid market fragmentation;

deleted

Or. fr

PE652.393v01-00 30/67 AM\1205570EN.docx

EN




deleted

Or. en




deleted

Or. en

AM\1205570EN.docx 31/67 PE652.393v01-00

EN

Amendment 62Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna CavazziniDraft opinionParagraph 7



7. Stresses that where public money contributes to the development or implementation of an algorithmic system, alongside open procurement and open contracting standards, the code, the generated data -as far as it is non-personal- and the trained model should be public by default, to enable transparency and reuse, among other goals, to maximise the achievement of the Single Market, and to avoid market fragmentation;

Or. en

Amendment 63Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Marc Angel, Adriana Maldonado López, Maria Grapini, Brando Benifei, Andreas SchiederDraft opinionParagraph 7



7. Stresses that where money originating from public sources contributes significantly to the development or implementation of an algorithmic system, the code, the generated data - as far as it is non-personal - and the trained model should be public by default, to enable transparency and reuse, among other goals, to maximise the achievement of the Single Market, and to avoid market fragmentation;

Or. en

PE652.393v01-00 32/67 AM\1205570EN.docx

EN




7. Stresses that where public money contributes to the development or implementation of an algorithmic system, the code, the generated data -as far as it is non-personal- and the trained model should be public by default to guarantee transparency and enable their reuse to foster innovation; stresses that in this way, the full potential of the Single Market can be unlocked avoiding market fragmentation;

Or. en

Amendment 65Maria da Graça Carvalho, Edina Tóth, Andrey KovatchevDraft opinionParagraph 7



7. Stresses that where public money contributes to the development or implementation of an algorithmic system, the code, the generated data -as far as it is non-personal- and the trained model should be accessible, to enable transparency and reuse, among other goals, to maximise the achievement of the Single Market, and to avoid market fragmentation;

Or. en



7. Stresses that where public money 7. Stresses that where public money

AM\1205570EN.docx 33/67 PE652.393v01-00

EN

contributes to the development or implementation of an algorithmic system, the code, the generated data -as far as it is non-personal- and the trained model should be public by default, to enable transparency and reuse, among other goals, to maximise the achievement of the Single Market, and to avoid market fragmentation;

contributes to the development or implementation of an algorithmic system, the generated data -as far as it is non-personal- and the trained model should be public by default, to enable transparency and reuse, among other goals, to maximise the achievement of the Single Market, and to avoid market fragmentation;

Or. en

Amendment 67Maria da Graça Carvalho, Róża Thun und Hohenstein, Edina Tóth, Andrey Kovatchev, Marion Walsmann, Pascal ArimontDraft opinionParagraph 7 a (new)


7a. Recalls that an examination of the current EU legal framework, including the consumer law acquis, data protection legislation, product liability legislation, product safety and market surveillance legislation, is needed to check that it is able to respond to the emergence of AI and automated decision-making and that it is able to provide a high level of consumer protection;

Or. en



7a. Stresses that the data sets and the processes that yield the AI system’s decision, including those of data gathering and data labelling as well as the algorithms used, should be documented to the best possible standard to allow for traceability and an increase in transparency; stresses that this also applies to the decisions made by the AI

PE652.393v01-00 34/67 AM\1205570EN.docx

EN

system;

Or. en

Amendment 69Christel Schaldemose, Leszek Miller, Evelyne Gebhardt, Marc Angel, Alex Agius Saliba, Sándor Rónai, Adriana Maldonado López, Maria Grapini, Andreas Schieder, Sylvie Guillaume, Brando BenifeiDraft opinionParagraph 7 a (new)


7a. Underlines that the increased use of artificial intelligence requires a strong focus on digital security, as the large amount of data creates new risks of cyberattacks; calls on the Commission to develop clear guidelines for businesses and public agencies to take the necessary precautions when using artificial intelligence;

Or. en



7a. Asks the Commission to issue binding rules for companies to publish transparency reports including the existence, functionality, process, main criteria, the logic behind, the datasets used and possible outcome of algorithmic systems and efforts to identify, prevent and mitigate discrimination in algorithmic systems in a timely, impartial, easily-readable, and accessible manner;

Or. en

AM\1205570EN.docx 35/67 PE652.393v01-00

EN



7a. Calls for a uniform implementation of the system of risk classification and related legal obligations to ensure a level-playing field among the Member States and to prevent a fragmentation of the internal market;

Or. en

Amendment 72Virginie Joron, Jean-Lin Lacapelle, Marco CampomenosiDraft opinionParagraph 7 a (new)


7a. Stresses that where European public funds contribute to the development or implementation of an algorithmic system of a consortium comprising a non-European company, the code and the non-personal data generated should be public by default.

Or. fr



7b. Underlines that data sets used by AI systems (both for training and operation) may suffer from the inclusion of inadvertent historic bias, incompleteness and bad governance models; stresses that the continuation of such biases could lead to unintended (in)direct prejudice and discrimination

PE652.393v01-00 36/67 AM\1205570EN.docx

EN

against certain groups or people, potentially exacerbating prejudice and marginalisation; notes that harm can also result from the intentional exploitation of (consumer) biases or by engaging in unfair competition, such as the homogenisation of prices by means of collusion or a non-transparent market; stresses that identifiable and discriminatory bias should be removed in the collection phase where possible; notes that the way in which AI systems are developed (e.g. algorithms’ programming) may also suffer from unfair bias; stresses that this could be counteracted by putting in place oversight processes to analyse and address the system’s purpose, constraints, requirements and decisions in a clear and transparent manner; notes that hiring from diverse backgrounds should be encouraged;

Or. en

Amendment 74Virginie Joron, Jean-Lin Lacapelle, Marco CampomenosiDraft opinionParagraph 7 b (new)


7b. Stresses that where non-European public funds contribute to the development or implementation in Europe of an algorithmic system, the code and the non-personal data generated should be public by default.

Or. fr

AM\1205570EN.docx 37/67 PE652.393v01-00

EN

Amendment 75Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Evelyne Gebhardt, Marc Angel, Adriana Maldonado López, Maria Grapini, Sándor Rónai, Brando Benifei, Andreas SchiederDraft opinion Paragraph 7 b (new)


7b. Considers that AI, IoT, and other emerging technologies have enormous potential to deliver opportunities for consumers to have access to several amenities in many aspects of their lives alongside with better products and services, as well as to benefit from better market surveillance, as long as all applicable principles, conditions (including transparency and auditability), and regulations continue to apply;

Or. en

Amendment 76Evelyne Gebhardt, Marc Angel, Monika Beňová, Alex Agius Saliba, Andreas Schieder, Maria Grapini, Sylvie Guillaume, Sándor RónaiDraft opinionParagraph 7 b (new)


7b. Calls for the establishment of an Union-wide registration system for artificial intelligence, robotics and related technologies to support the uniform and transparent implementation of the risk classification in the Union;

Or. en



8. Underlines the importance of ensuring that the interests of marginalised

8. Underlines the importance of ensuring that the interests of all consumers

PE652.393v01-00 38/67 AM\1205570EN.docx

EN

and vulnerable consumers and groups are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

and groups, including marginalised and vulnerable, are adequately taken into account; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Or. en

Amendment 78Jordi Cañas, Svenja HahnDraft opinionParagraph 8


8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and groups are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and groups, such as persons with disabilities, are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Or. en

AM\1205570EN.docx 39/67 PE652.393v01-00

EN




8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and groups are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection, intellectual property rights and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Or. en

Amendment 80Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-MarquesDraft opinionParagraph 8


8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and groups are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and

8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and groups are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, where possible via APIs, while fully respecting

PE652.393v01-00 40/67 AM\1205570EN.docx

EN

privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Or. en

Amendment 81Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Marc Angel, Alex Agius Saliba, Adriana Maldonado López, Maria Grapini, Brando Benifei, Andreas SchiederDraft opinionParagraph 8



8. Underlines the importance of ensuring that the interests of consumers and groups which are marginalised or in vulnerable situations are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Or. en

Amendment 82Svenja Hahn, Claudia Gamon, Dita Charanzová, Sandro Gozi, Stéphanie Yon-CourtinDraft opinionParagraph 8


8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and groups are

8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and groups are

AM\1205570EN.docx 41/67 PE652.393v01-00

EN

adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

adequately taken into account and represented in any future regulatory framework of ethical aspects; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Or. en




8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and groups are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of educating consumers to be more informed when dealing with algorithmic systems in order to protect them from potential risks and uphold their rights;

Or. en

Amendment 84Virginie Joron, Jean-Lin Lacapelle, Marco Campomenosi

PE652.393v01-00 42/67 AM\1205570EN.docx

EN




8. Underlines the importance of ensuring that the interests of marginalised and vulnerable consumers and European citizens are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably university or scientific researchers, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Or. fr

Amendment 85Maria da Graça Carvalho, Edina Tóth, Marion Walsmann, Pascal ArimontDraft opinionParagraph 8

AM\1205570EN.docx 43/67 PE652.393v01-00

EN



8. Underlines the importance of ensuring that the interests of consumers in vulnerable situations are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to non-personal data could be extended to notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; recalls the importance of training and giving basic skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

Or. en




8. Underlines the importance of ensuring that the interests of consumers are adequately taken into account and represented in any future regulatory framework; notes that for the purpose of analysing the impacts of algorithmic systems on consumers, access to data should be extended to appropriate parties notably independent researchers, media and civil society organisations, while fully respecting Union data protection, privacy law and business secrecy rules; recalls the importance of training and giving basic digital skills to consumers to deal with algorithmic systems in order to protect them from potential risks and detriment of their rights;

PE652.393v01-00 44/67 AM\1205570EN.docx

EN

Or. en

Amendment 87Maria-Manuel Leitão-Marques, Christel Schaldemose, Evelyne Gebhardt, Marc Angel, Adriana Maldonado López, Maria Grapini, Andreas SchiederDraft opinionParagraph 8 a (new)


8a. Considers that, when it comes to filling the gaps on the practical implementation of ethical guidelines on algorithmic systems and connected technologies, models such as the so-called VCIO model (Values, Criteria, Indicators, Observables) should be studied and evaluated in terms of fitness for purpose; further considers that such models to concretise and implement AI system requirements, as well as to produce labels that allow companies to communicate the ethical properties of their products clearly and uniformly through a standardised risk matrix, could enhance consumer literacy, information, and awareness;

Or. en

Amendment 88Clara Ponsatí ObiolsDraft opinionParagraph 8 a (new)


8a. Stresses the importance of ensuring that the collective rights of national and linguistic minorities are taken into account and represented in any future regulatory framework, in particular in relation to the regional and minority languages of the Union;

Or. en

Amendment 89Maria da Graça Carvalho, Andrey Kovatchev, Róża Thun und Hohenstein, Pascal Arimont

AM\1205570EN.docx 45/67 PE652.393v01-00

EN



9. Underlines the importance of training highly skilled professionals in this area and ensuring the mutual recognition of such qualifications across the Union;

9. Underlines the importance of training highly skilled professionals in this area, the need of having diverse teams of developers and engineers working alongside with key actors to prevent gender and cultural bias of being inadvertently included in AI algorithms, systems and applications and ensure the mutual recognition of such qualifications across the Union;

Or. en

Amendment 90Kateřina KonečnáDraft opinionParagraph 9



9. Underlines the importance of training highly skilled professionals in this area and ensuring the mutual recognition of such qualifications across the Union; supports the creation of educational curricula and public awareness activities around the societal, legal, and ethical impact of AI;

Or. en




9. Underlines the importance of achieving a high-level of overall digital literacy and of training highly skilled professionals in this area as well as ensuring the mutual recognition of such

PE652.393v01-00 46/67 AM\1205570EN.docx

EN

qualifications across the Union;

Or. en




9. Underlines the importance of upskilling and training highly skilled professionals in this area and ensuring the mutual recognition of such qualifications across the Union;

Or. en



9a. Calls on the Commission to promote and fund the development of human-centric artificial intelligence, robotics and related technologies that address environment and climate challenges and that ensure equal access to and enjoyment of fundamental rights through the use of tax, procurement, or other incentives;

Or. en

Amendment 94Kateřina KonečnáDraft opinionParagraph 9 a (new)


9a. Notes that those who own or operate inputs to AI systems and profit

AM\1205570EN.docx 47/67 PE652.393v01-00

EN

from it should be asked to help fund the development of AI literacy programs for consumers as this is in the best interest of both the company and society as a whole;

Or. en



9b. Notes that particular attention in AI literacy programs must also be paid to situations where AI systems can cause or exacerbate adverse impacts due to asymmetries of power or information, such as between employers and employees, businesses and consumers or governments and citizens;

Or. en

Amendment 96Alexandra Geeseon behalf of the Greens/EFA GroupRóża Thun und Hohenstein, Kim Van Sparrentak, Petra De Sutter, Anna CavazziniDraft opinionParagraph 9 b (new)


9b. Underlines that artificial intelligence and algorithmic systems should be legally compliant, robust, reliable and secure by design; calls on the Commission to ensure that the Union’s regulatory approach to algorithmic systems includes appropriate measures to enable that these systems are subject to independent control and oversight;

Or. en

Amendment 97Virginie Joron, Jean-Lin Lacapelle, Marco Campomenosi, Alessandra Basso

PE652.393v01-00 48/67 AM\1205570EN.docx

EN



10. Calls for the Union to establish a European market surveillance structure for algorithmic systems issuing guidance, opinions and expertise to Member States’ authorities;

deleted

Or. fr




deleted

Or. en

Amendment 99Evelyne Gebhardt, Marc Angel, Monika Beňová, Alex Agius Saliba, Andreas Schieder, Sylvie Guillaume, Sándor Rónai, Brando BenifeiDraft opinionParagraph 10



10. Calls for the establishment of a European market surveillance structure for algorithmic systems in the framework of a European Agency for Artificial Intelligence, which the Parliament proposed in its resolution of 16 February 20171a, issuing guidance, opinions and expertise to Member States’ authorities, monitoring the implementation of relevant EU legislation, addressing potential consumer protection issues, identifying standards for best practice, and, where appropriate, making

AM\1205570EN.docx 49/67 PE652.393v01-00

EN

recommendations for regulatory measures;

__________________1a European Parliament resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics (2015/2103(INL)) (OJ C 252, 18.7.2018, p. 239).

Or. en

Amendment 100Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Marc Angel, Adriana Maldonado López, Andreas SchiederDraft opinionParagraph 10



10. Calls for the Union to consider the establishment of a European market surveillance structure for algorithmic systems issuing guidance, opinions and expertise to Member States’ authorities; further calls for this structure to be appropriately advised by stakeholder organisations (such as consumer protection organisations) in order to ensure wide consumer representation;

Or. en

Amendment 101Alexandra Geeseon behalf of the Greens/EFA GroupRóża Thun und Hohenstein, Kim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-MarquesDraft opinionParagraph 10



10. Calls on the Commission to establish a European market surveillance structure for algorithmic systems issuing guidance, opinions and expertise to Member States’ authorities; considers that due to the disproportionate impact of

PE652.393v01-00 50/67 AM\1205570EN.docx

EN

algorithmic systems on women and minorities, the decisional levels of such a structure should be diverse and gender balanced;

Or. en

Amendment 102Christel Schaldemose, Leszek Miller, Evelyne Gebhardt, Marc Angel, Alex Agius Saliba, Sándor Rónai, Adriana Maldonado López, Maria Grapini, Andreas Schieder, Sylvie Guillaume, Brando BenifeiDraft opinionParagraph 10



10. Calls for the Union to establish a European market surveillance structure for algorithmic systems issuing guidance, opinions and expertise to Member States’ authorities; emphasizes that Member States must develop risk-management strategies for AI in the context of their national market surveillance strategies.

Or. en

Amendment 103Jordi Cañas, Svenja HahnDraft opinionParagraph 10



10. Calls for the Commission to promote the exchange of information related to algorithmic systems between market surveillance authorities, and to support the development of a common understanding in the Single Market by issuing guidance, opinions and expertise to Member States’ authorities;

Or. en

AM\1205570EN.docx 51/67 PE652.393v01-00

EN

Amendment 104Maria da Graça Carvalho, Edina Tóth, Andrey Kovatchev, Marion WalsmannDraft opinionParagraph 10



10. Calls for the Union to establish a European market surveillance structure composed of national market surveillance authorities issuing guidance, opinions and expertise to Member States’ authorities;

Or. en




10. Calls for the use of existing bodies on either Union or member state level to conduct market surveillance for algorithmic systems issuing guidance, opinions and expertise to Member States’ authorities;

Or. en



10a. Calls on the Commission and the authorities of the Member States to combat tax evasion and the abuse of a dominant position by non-European companies in Europe active in the market for algorithmic systems;

Or. fr

Amendment 107Evžen Tošenovský

PE652.393v01-00 52/67 AM\1205570EN.docx

EN



11. Notes that it is essential for the software documentation, the algorithms and data sets used to be fully accessible to market surveillance authorities, while respecting Union law; invites the Commission to assess if additional prerogatives should be given to market surveillance authorities in this respect;

11. Notes that as regards the high risk applications, it is essential for the software documentation, the algorithms and data sets used to be fully accessible to market surveillance authorities designated by Member States, while respecting Union law;

Or. en

Amendment 108Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Adriana Maldonado López, Maria Grapini, Sándor Rónai, Brando Benifei, Andreas SchiederDraft opinionParagraph 11



11. Notes that it is essential for the software documentation, the algorithms and data sets used or produced by artificial intelligence, robotics, and related technologies to be fully accessible to market surveillance authorities, while respecting Union law; further notes that such elements should be preserved by those who are involved in the different stages of the development of algorithmic systems and in proportion of their liability, namely through distributed ledger technologies, such as block-chain; invites the Commission to assess if additional prerogatives should be given to market surveillance authorities in this respect;

Or. en

Amendment 109Jordi CañasDraft opinionParagraph 11

AM\1205570EN.docx 53/67 PE652.393v01-00

EN



11. Notes that the software documentation, algorithms and data sets should be explainable and accessible to market surveillance authorities to the greatest possible extent, in accordance with Union law and particularly a future regulatory framework of ethical aspects; invites the Commission to assess if additional prerogatives should be given to market surveillance authorities in this respect; considers that an examination of the current market surveillance legislation might be necessary to avoid its obsolescence and ensure that it responds ethically to the emergence of artificial intelligence, robotics and related technologies;

Or. en

Amendment 110Maria da Graça Carvalho, Edina Tóth, Róża Thun und Hohenstein, Marion Walsmann, Pascal ArimontDraft opinionParagraph 11



11. Notes that it is essential for the software documentation, the algorithms and data sets used to be explainable to market surveillance authorities, while respecting Union law; invites the Commission to assess if additional prerogatives should be given to market surveillance authorities in this respect;

Or. en


PE652.393v01-00 54/67 AM\1205570EN.docx

EN



11. Notes that it is essential for the software documentation, the algorithms and data sets used to be accessible to market surveillance authorities, while respecting Union law and trade secrets; invites the Commission to assess if additional prerogatives should be given to market surveillance authorities in this respect;

Or. en

Amendment 112Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-Marques, Svenja HahnDraft opinionParagraph 11



11. Notes that it is essential for the risk assessment documentation, the software documentation, the algorithms and data sets used to be fully accessible to market surveillance authorities, while respecting Union law; notes that additional prerogatives should be given to market surveillance authorities in this respect;

Or. en

Amendment 113Alexandra Geeseon behalf of the Greens/EFA GroupRóża Thun und Hohenstein, Kim Van Sparrentak, Petra De Sutter, Anna Cavazzini, Maria-Manuel Leitão-MarquesDraft opinionParagraph 12


12. Calls for the designation by each Member State of a competent national authority for monitoring the application of

12. Calls for the designation and sufficient funding by each Member State of a competent national authority for

AM\1205570EN.docx 55/67 PE652.393v01-00

EN

the provisions; monitoring the application of the provisions related to algorithmic systems;

Or. en

Amendment 114Clara Ponsatí ObiolsDraft opinionParagraph 12


12. Calls for the designation by each Member State of a competent national authority for monitoring the application of the provisions;

12. Calls for the designation by each Member State of a competent national authority or of several competent regional authorities for monitoring the application of the provisions;

Or. en




12. Calls for the designation by each Member State of a competent national authority for monitoring the application of the provisions related to algorithms;

Or. en

Amendment 116Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Evelyne Gebhardt, Adriana Maldonado López, Maria Grapini, Brando Benifei, Andreas SchiederDraft opinionParagraph 12



12. Calls for the designation by each Member State of a competent national enforcement body(NEB) for monitoring the application of the provisions;

PE652.393v01-00 56/67 AM\1205570EN.docx

EN

Or. en



12a. Stresses the need for the authorities to put an end to the practices of massive and systematic profiling or tracking of consumers organised by some players in the advertising industry on the internet or mobile phones; notes that these tools collect very sensitive data about behaviour, interests or health, even from minors and that they are then used by third parties; regrets that the lack of supervision by the authorities strengthens the market power of less ethical players and dissuades some players from offering more ethical solutions that are more favourable to consumers.

Or. fr

Amendment 118Virginie Joron, Jean-Lin Lacapelle, Alessandra Basso, Marco CampomenosiDraft opinionParagraph 13


13. Calls for the establishment of a European market surveillance board for algorithmic systems, to ensure a level playing field and to avoid fragmentation of the internal market, to decide with a qualified majority and by secret vote in case of different decisions on algorithmic systems used in more than one Member State, as well as at the request of the majority of the national authorities;

deleted

Or. fr

Amendment 119Dita Charanzová

AM\1205570EN.docx 57/67 PE652.393v01-00

EN




deleted

Or. en




13. Calls on the Commission to ensure that Union legislation related to algorithmic systems is enforced in all Member States to avoid fragmentation of the internal market;

Or. en

Amendment 121Jordi CañasDraft opinionParagraph 13


13. Calls for the establishment of a European market surveillance board for algorithmic systems, to ensure a level playing field and to avoid fragmentation of

13. Calls for improved and effective cooperation of national surveillance bodies as regards algorithmic systems across the Union, as well as harmonised

PE652.393v01-00 58/67 AM\1205570EN.docx

EN

the internal market, to decide with a qualified majority and by secret vote in case of different decisions on algorithmic systems used in more than one Member State, as well as at the request of the majority of the national authorities;

risk management strategies for artificial intelligence, in order to ensure a level playing field and to avoid fragmentation of the internal market;

Or. en

Amendment 122Maria da Graça Carvalho, Róża Thun und Hohenstein, Edina Tóth, Marion WalsmannDraft opinionParagraph 13



13. Calls for the establishment of a European market surveillance board for algorithmic systems, to ensure a level playing field and to avoid fragmentation of the internal market;

Or. en




13. Calls for a strong coordination of Member State’s authorities, for instance through instruments such as a European market surveillance board for algorithmic systems, to ensure a level playing field and to avoid fragmentation of the internal market;

AM\1205570EN.docx 59/67 PE652.393v01-00

EN

Or. en




13. Calls for the establishment of a European market surveillance advisory committee for algorithmic systems, composed of the representatives of the national competent authorities, the Commission and relevant EU agencies such as ENISA, to contribute to a level playing field and to avoiding fragmentation of the internal market by issuing guidelines and recommendations to Member States’ authorities;

Or. en

Amendment 125Evelyne Gebhardt, Marc Angel, Monika Beňová, Andreas Schieder, Maria Grapini, Sylvie Guillaume, Sándor Rónai, Brando BenifeiDraft opinionParagraph 13



13. Calls for the establishment of a European market surveillance board for algorithmic systems in the framework of a European Agency for Artificial Intelligence, to ensure a level playing field and to avoid fragmentation of the internal market, to decide with a qualified majority and by secret vote in case of different decisions on algorithmic systems used in more than one Member State, as well as at the request of the majority of the national authorities;

Or. en

Amendment 126Clara Ponsatí Obiols

PE652.393v01-00 60/67 AM\1205570EN.docx

EN




13. Calls for the establishment of a European market surveillance board for algorithmic systems, to ensure a level playing field and to avoid fragmentation of the internal market, to decide with a qualified majority and by secret vote in case of different decisions on algorithmic systems used in more than one Member State, as well as at the request of the majority of the national or regional authorities;

Or. en

Amendment 127Evžen TošenovskýDraft opinionParagraph 13 a (new)


13a. Acknowledges valuable outputs of the High-Level Expert Group on Artificial Intelligence, particularly ‘The Ethics Guidelines for Trustworthy Artificial Intelligence’; suggests that this group comprising representatives from academia, civil society and industry, as well as the European AI Alliance, might provide expertise to the European market surveillance advisory committee for algorithmic systems;

Or. en

Amendment 128Evelyne Gebhardt, Marc Angel, Monika Beňová, Alex Agius Saliba, Andreas Schieder, Maria Grapini, Sylvie Guillaume, Sándor RónaiDraft opinionParagraph 13 a (new)

AM\1205570EN.docx 61/67 PE652.393v01-00

EN


13a. Believes that a European certification of ethical compliance should be designed in such a way as to inform consumers about the risk level of a product or a service with an algorithmic component as well as its trustworthiness in the light of the ethical principles and all other requirements based on relevant Union legislation;

Or. en

Amendment 129Maria-Manuel Leitão-Marques, Christel Schaldemose, Sylvie Guillaume, Evelyne Gebhardt, Marc Angel, Alex Agius Saliba, Adriana Maldonado López, Maria Grapini, Sándor Rónai, Brando Benifei, Andreas SchiederDraft opinionParagraph 13 a (new)


13a. Stresses the need for national market surveillance authorities to be reinforced in terms of capacity, skills, and competences in AI, IoT and other related technologies, as well as knowledge about its specific risks;

Or. en

Amendment 130Virginie Joron, Jean-Lin Lacapelle, Marco CampomenosiDraft opinionParagraph 13 a (new)


13a. Demands an update of European competition and public procurement law to promote the emergence of world-class players based in Europe;

Or. fr

PE652.393v01-00 62/67 AM\1205570EN.docx

EN

Amendment 131Virginie Joron, Jean-Lin Lacapelle, Alessandra Basso, Marco CampomenosiDraft opinionParagraph 13 b (new)


13b. Calls for measures to encourage global companies to relocate part of their employment, research and digital systems and products to Europe, if they want to benefit from full access to the European market;

Or. fr

Amendment 132Virginie Joron, Jean-Lin Lacapelle, Alessandra Basso, Marco CampomenosiDraft opinionParagraph 13 c (new)


13c. Calls for respect for European and national preferences, for the development of European territories and employment in the sector of artificial intelligence and robotics; stresses the importance of preventing the takeover of strategic companies by non-European players;

Or. fr

Amendment 133Kateřina KonečnáDraft opinionSubheading 5a (new)


Equality and non-discrimination

Or. en

Amendment 134Kateřina Konečná, Anne-Sophie PelletierDraft opinionParagraph 13 d (new) (after Subheading 5 a new)

AM\1205570EN.docx 63/67 PE652.393v01-00

EN


13d. Highlights the need for equal respect for the moral worth and that dignity of all human beings must be ensured. This goes beyond non-discrimination, which tolerates the drawing of distinctions between dissimilar situations based on objective justifications. In an AI context, equality entails that the system’s operations cannot generate unfairly biased outputs (e.g. the data used to train AI systems should be as inclusive as possible, representing different population groups).; calls for adequate protection for potentially vulnerable persons and groups, such as workers, women, persons with disabilities, ethnic minorities, elderly people, children, consumers or others at risk of exclusion;

Or. en

Amendment 135Kateřina KonečnáDraft opinionParagraph 13 e (new) (after Subheading 5 a new)


13e. Notes that, particularly in business-to-consumer domains, systems should be user-centric and designed in a way that allows all people to use AI products or services, regardless of their age, gender, abilities or characteristics. Accessibility to this technology for persons with disabilities, which are present in all societal groups, is of particular importance. AI systems should not have a one-size-fits-all approach and should consider Universal Design principles addressing the widest possible range of users, following relevant accessibility standards. This will enable equitable access and active participation of all people in existing and emerging

PE652.393v01-00 64/67 AM\1205570EN.docx

EN

computer-mediated human activities and with regard to assistive technologies;

Or. en

Amendment 136Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna CavazziniDraft opinionParagraph 14 a (new)


14a. The main principles and aims of the proposal are:

1. A genuine Digital Single Market requires full harmonisation by a Regulation.

2. For algorithmic systems that are considered of a higher risk, horizontally applicable safeguards are essential, irrespective of sector-specific rules.

3. The horizontal Regulation should build on and ensure the implementation of the European framework of rights of users and consumers, in particular the protection of privacy, non-discrimination, dignity, fairness, freedom of expression and the right to an effective remedy.

Or. en

Amendment 137Alexandra Geeseon behalf of the Greens/EFA GroupKim Van Sparrentak, Petra De Sutter, Anna CavazziniDraft opinionParagraph 14 b (new)


14b. The determination of the risk of algorithmic systems in a future horizontal regulation on artificial intelligence should be based on a combination of the severity of the potential damage and the

AM\1205570EN.docx 65/67 PE652.393v01-00

EN

probability of its occurrence. An increased risk potential of an algorithmic system must be accompanied by a higher degree of regulatory intervention. For the algorithmic systems in the lowest risk category, no further legal obligations should apply. A pre-defined set of criteria should be used to define the risk potential of an application or a scope of application. These criteria should include:

1. The quality and integrity of the collected and processed data

2. The intensity of the possible violation of rights, by taking into account:

(a) the depth of the potential damage caused;

(b) the number of persons affected;

(c) the sum of potential damages, since on the one hand, there may be particularly intensive individual cases of rights violations and, on the other hand, there may be particularly frequent minor rights violations whose sum nevertheless has social relevance.

3. The impact on fundamental rights and freedoms;

4. The impact on society and the environment, including:

(a) material/monetary damage and welfare effects for individuals or groups, for example in the allocation of resources or access to markets;

(b) social implications, such as social, psychological, cultural and economic dimensions;

(c) the systemic relevance for the functioning of society and democracy, such as elections, the formation of public opinion or the creation of socially useful infrastructure;

(d) the impact on environmental sustainability, such as its energy consumption, the use of raw materials or

PE652.393v01-00 66/67 AM\1205570EN.docx

EN

effects on the climate.

5. The likelihood of the occurrence of damage including:

(a) the role of the algorithm in taking the decision;

(b) the dependence of the persons impacted, for example private sector services in competitive environments versus public services provided by the state or services by a market dominant company;

(c) the reversibility of the consequences of a decision.

Or. en

AM\1205570EN.docx 67/67 PE652.393v01-00

EN

Documents

› doceo › document › I… · Web viewWhereas there are serious concerns that the current Union legal framework, including the consumer law acquis, data protection legislation,