Cinco formas de discriminación algorítmica: Hacia una descripción material

• Autores: Erik Francesc Obiol Anaya, Axell Cervera
• Localización: Indret: Revista para el Análisis del Derecho, ISSN-e 1698-739X, Nº. 4, 2025
• Idioma: español
• DOI: 10.31009/InDret.2025.i4.17
• Enlaces
  • Texto completo
• Resumen

  • En los últimos años, la automatización de procesos a través de mecanismos algorítmicos ha experimentado un incremento significativo en sectores como la salud, la educación, el sistema crediticio, el entorno laboral, la vigilancia policial, entre otros. Aunque es evidente que dicha transformación ha traído importantes beneficios en términos de eficiencia y reducción de costos, la implementación de estos sistemas no está exenta de riesgos sociales, en tanto poseen la capacidad de generar dinámicas discriminatorias con un nivel de complejidad mayor a la observada en humanos, lo cual dificulta su comprensión técnica y abordaje analítico. Por este motivo, en el presente trabajo de investigación se examinaron cinco formas en las que puede configurarse la discriminación algorítmica; a saber, discriminación por correlaciones sesgadas, por variables sustitutas, por grupos algorítmicos, por desajuste algorítmico y por bucle de retroalimentación negativo. Posterior a ello, se estableció una propuesta de carácter descriptivo que, por un lado, permite explicar cómo se materializa este fenómeno en la realidad observable, y por el otro, plantea un punto de partida conceptual para futuras investigaciones sobre el tema.

• Referencias bibliográficas
  • ADAM,Georgeet al., «Hidden Risks of Machine Learning Applied to Healthcare: Unintended Feedback Loops Between Models and Future...
  • ADAMS-PRASSL,Jeremias et al.,«Directly Discriminatory Algorithms», The Modern Law Review, 2023, pp. 144-175.
  • ALBAROUDI,Elham et al.,«A Comprehensive Review of AI Techniques for Addressing Algorithmic Bias in Job Hiring», AI, vol. 5, 2024, pp. 383-404.
  • ALELYANI,Salem,«Detection and Evaluation of Machine Learning Bias», Applied Sciences, 2021, pp. 1-17.
  • ALFAIF,Yousef,«Recommender Systems Applications: Data Sources, Features, and Challenges», Information, 2024, pp. 1-25.
  • ANDRINGA,Sible/GODFROID,Aline,«Sampling Bias and the Problem of Generalizability in Applied Linguistics», Annual Review of Applied Linguistics,...
  • ARAUJO,Theo et al.,«In AI we trust? Perceptions about automated decision making by artificial intelligence», AI & Society, 2020, pp. 611-623.
  • BAJRACHARYA,Aakriti et al.,«Recent Advances in Algorithmic Biases and Fairness in Financial Services: A Survey», 2022, pp. 1-15.
  • BAROCAS,Solon et al.,«Fairness and Machine Learning. Limitations and Opportunities», 2023, pp. 1-284.
  • BAUER, Kevin et al.,«Feedback Loops in Machine Learning: A Study on the Interplay of Continuous Updating and Human Discrimination»,...
  • BINNS,Reuben,«Human Judgment in Algorithmic Loops: Individual Justice and Automated Decision-Making», Regulation & Governance,...
  • BLOUNT,Kelly, «Applying The Presumption Of Innocence To Policing With AI», 2021, pp. 33-48.
  • BOLSON RUZZARIN,Mateus, «Conversation with Noam Chomsky - The Responsibility of Intellectuals», 2021. Disponible en: https://www.youtube.com/watch?v=7M35aasejgI&t=78s
  • BORGSTEDE,Matthias/EGGERT,Frank,«Squaring the Circle: From Latent Variables to Theory-Based Measurement», Theory & Psychology,...
  • BRIGHT, Jonathan et al.. «Generative AI is Already Widespread in the Public Sector», 2024, pp. 1-10.
  • CABIDDU,Franccesca et al.,«Why do Users Trust Algorithms? A Review and Conceptualization of Initial Trust and Trust Over Time», European Management...
  • CHAKRABORTY,Joymallya et al.,«Bias in Machine Learning Software: Why? How? What to Do?», Proceedings of the 29th ACM Joint European Software...
  • CHEN,Mo/GROSSKLAGS,Jens, «Social Control in the Digital Transformation of Society: A Case Study of the Chinese Social Credit System»,...
  • CHEN,Zhenpeng,«A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifiers», ACM Transactions...
  • CHINTA,Sribala et al.,«FairAIED: Navigating Fairness, Bias, and Ethics in Educational AI Applications», 2024, pp, 1-47.
  • CIRCIUMARU,Alexandru,«Futureproofing EU Law. The Case of Algorithmic Discrimination», University of Oxford, 2021, pp. 1-123.
  • CROSS,James,«Bias in Medical AI: Implications for Clinical Decision-Making», PLOS Digital Health, vol. 3(11), 2024, pp. 1-19.
  • DASTIN,Jeffrey,«Amazon Scraps Secret AI Recruiting Tool that Showed Bias Against Women», Reuters, 2018. Obtenido de: https://www.reuters.com/article/world/insight-amazon-scraps-secret-ai-recruiting-tool-that-...
  • LATA,Mateusz et al.,«A Sociotechnical View of Algorithmic Fairness», Information Systems Journal, 2021, pp. 1-87.
  • ENGSTROM,David et al.,«Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies», 2020, pp. 1-122.
  • FARIČ,Ana/BRATKO, Ivan , «Machine Bias: A Survey of Issues», Informatica, núm. 48, 2024, pp. 205–212.
  • FORAN,Michael,«Grounding Unlawful Discrimination», Legal Theory, vol. 28, 2022, pp. 3-34.
  • FORD,Christopher et al.,«Racial Differences in Performance of HbA1c for the Classification of Diabetes and Prediabetes among...
  • GAMA,Ricardo et al., «Multi-Agent Environments for Vehicle Routing Problems», 2024., pp. 1-17.
  • GARBELLINI FILHO,Luiz, «El enfrentamiento a la discriminación interseccional en el sistema interamericano de derechos humano....
  • GHAFFARY,Shirin,«How TikTok’s Hate Speech Detection Tool Set Off a Debate About Racial Bias on the App», Vox, 2021. https://www.vox.com/recode/2021/7/7/22566017/tiktok-black-creators-ziggi-tyler-debate-about-black-lives-matter-racial-bias-social-media
  • GRANT,Nico/HILL,Kashmir,«Google’s Photo App Still Can’t Find Gorillas. And Neither Can Apple’s», The New York Times, 2023....
  • GREEN,Ben,«The Flaws of Policies Requiring Human Oversight of Government Algorithms», Computer Law & Security Review, 2022,...
  • HAN,Xudong et al.,«Balancing out Bias: Achieving Fairness Through Balanced Training», 2022, pp. 1-16.
  • HASAN,Emrul et al.,«Review-based Recommender Systems: A Survey of Approaches, Challenges and Future Perspectives», Proceedings of the...
  • HEAVEN,Will,«Predictive Policing Algorithms Are Racist. They Need to Be Dismantled», MIT Technology Review, 2020. https://www.technologyreview.com/2020/07/17/1005396/predictive-policing-algorithms-racist-dismantled-machine-learning-bias-criminal-justice/
  • HINDER,Fabian et al.,«One or Two Things We Know About Concept Drift. A Survey on Monitoring Evolving Environments», 2023, pp. 1-44.
  • HIREVUE,«About Us», Hirevue, 2025. Obtenido de: https://www.hirevue.com/about
  • HOVAKIMYAN,Gurgen/BRAVO,Jorge,«Evolving Strategies in Machine Learning: A Systematic Review of Concept Drift Detection», Information,...
  • HU,Lily,«What is “Race” in Algorithmic Discrimination on the Basis of Race?», Journal of Moral Philosophy, 2023, pp. 1-23.
  • IMANA,Basileal et al.,«Auditing for Discrimination in Algorithms Delivering Job Ads», Proceedings of the Web Conference...
  • INNERARITY, Daniel,«Justicia Algorítmica y Autodeterminación Deliberativa», IsegoríaRevista de Filosofía moral y política, núm. 68, 2023,...
  • IRIONDO,Roberto,«Amazon Scraps Secret AI Recruiting Engine that Showed Biases Against Women», Carnegie Mellon University,2018....
  • KELLY-LYTH,Aislinn,«Algorithmic Discrimination at Work», European Labour Law Journal, vol. 14(2), 2023, pp. 152-171.
  • KIESLICH,Kimon/LÜNICH,Marco,«Regulating AI-Based Remote Biometric Identification. Investigating the Public Demand for Bans, Audits,...
  • KING,OWEN/MERTENS,Mayli,«Self Fulfilling Prophecy in Practical and Automated Prediction», Ethical Theory and Moral Practice, 2023,...
  • KÖCHLING, Alina/WEHNER,Marius, «Discriminated by an Algorithm: A Systematic Review of Discrimination and Fairness by Algorithmic...
  • LANDA ARROYO,Cesar,«El Derecho Fundamental a la Igualdad y No Discriminación en la Jurisprudencia del Tribunal Constitucional del...
  • MALIK,Esraa et al., «Credit Card Fraud Detection Using a New Hybrid Machine Learning Architecture», Mathematics, 2022, pp, 1-16.
  • MATA SIERRA,María,«La Discriminación por Indiferenciación y su Incidencia en el Ámbito Tributario», Revista Jurídica de la Universidad...
  • MEHRABI,Ninareh et al.,«A Survey on Bias and Fairness in Machine Learning», ACM Computing Surveys (CSUR), 2022, pp. 1-35.
  • MÖKANDER,Jakob et al,«Ethics Based Auditing of Automated Decision Making Systems: Nature, Scope, and Limitations», Science and Engineering...
  • MUÑOZ,GUTIÉRREZ, Catherine,«La Discriminación en una Sociedad Automatizada: Contribuciones desde América Latina», Revista...
  • NITTLE,Nadra,«Spend “Frivolously” and Be Penalized Under China’s New Social Credit System», Vox, 2018. Obtenido de: https://www.vox.com/the-goods/2018/11/2/18057450/china-social-credit-score-spend-frivolously-video-games
  • NOWAK,Marcin,«The Impact of Rule Based Decision Engines on Business Efficiency», Higson, 2024. Obtenido de: https://www.higson.io/blog/the-impact-of-rule-based-decision-engines-on-business-efficiency
  • PÁEZ,Andrés,«Negligent Algorithmic Discrimination», Law and Contemporary Problems, 2021, pp. 19-3 3.
  • PAGAN,Nicolò et al.,«A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems», 2023,...
  • PAPADOPOULOS,Petros et al.«A Systematic Review of Technologies and Standards Used in the Development of Rule Based Clinical Decision...
  • PAVLIDIS,Georgios,«Unlocking the Black Box: Analysing the EU Artificial Intelligence Act’s Framework for Explainability in AI»,...
  • PESSACH,Dana/SHMUELI,Erez,«Algorithmic Fairness», 2020, pp. 1-31.
  • PLATKIN,Matthew/IYER,Sundeep,«Guidance on Algorithmic Discrimination and the New Jersey Law Against Discrimination», 2025, p. 10.
  • PRINCE,Anya/SCHWARCZ,Daniel, «Proxy Discrimination in the Age of Artificial Intelligence and Big Data», Iowa Law Review, 2020, pp. 1257-1318.
  • RAMÍREZ-BUSTAMANTE, Natalia/PÁEZ,Andrés,. «Análisis Jurídico de la Discriminación Algorítmica en los Procesos de Selección Laboral», 2021,...
  • RATHGEB,Christian et al.,«Demographic Fairness in Biometric Systems: What Do the Experts Say?», 2023, pp. 1-9.
  • RAVISHANKAR,Pavan et al., «Provable Detection of Propagating Sampling Bias in Prediction Models», The Thirty-Seventh AAAI Conference...
  • ROAAVELLA,Marcela del Pilar et al., «Uso del Algoritmo COMPAS en el Proceso Penal y losRiesgos a los Derechos Humanos», RevIsta...
  • SALOMÉ RESURRECCIÓN,liliana, «La discriminación y algunos de sus calificativos: directa, indirecta, por indiferenciación,...
  • SAUNDERS,Natasha, «Security, digital border technologies, and immigration admissions: Challenges of and to non-discrimination,...
  • SCHWARTING,Rena/ULBRICHT,Lena. «Why Organization Matters in “Algorithmic Discrimination”», Köln Z Soziol, 2022, pp. 307-330.
  • SHAHBAZI,Nima,«Representation Bias in Data: A Survey on Identification and Resolution Techniques», Woodstock ’18: ACM Symposium...
  • SINGLA,Alex et al.,«The state of AI in early 2024: Gen AI adoption spikes and starts to generate value», QuantumBlack AI by McKinsey, 2024,...
  • SÓLMUNDSDÓTTIR, Agnes et al.,«Mean Machine Translations: On Gender Bias in Icelandic Machine Translations», Proceedings of the...
  • STARKE,Christopher et al.,«Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature», Big Data...
  • STRAUB, Vincent et al.,«Artificial intelligence in government: Concepts, standards, and a unified framework», Computer Science, 2023, pp....
  • SURESH,Harini et al.,«Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making», 12th ACM Conference...
  • TAPIALARA, Francisco, «A Conceptual Framework for Simulating Feedback Loops in Engineering Design», The University of Leeds, 2023, pp. 1-...
  • TEO,Sue Anne,«Artificial intelligence and its ‘slow violence’ to human rights», AI and Ethics, 2024, pp. 1-16.
  • TOMASEV,Nenad et al., «Manifestations of xenophobia in AI systems», AI & Society, vol. 40, 2025, pp. 741-763.
  • WACHTER,Sandra,«Theory of Artificial Immutability: Protecting Algorithmic Groups Under Anti-Discrimination Law», Tulane Law Review, 2022,...
  • WAHID,Sumra, «How to Get Away With Discrimination: The Use of Algorithms to Discriminate in the Internet Entertainment Industry», American...
  • WEERTS,Hilde et al., «Unlawful Proxy Discrimination: A Framework for Challenging Inherently Discriminatory Algorithms», 2024 ACM Conference...
  • WEIZENBAUM-INSTITUT., «Sandra Wachter - The Theory of Artificial Immutability», 2023. Disponible en: https://www.youtube.com/watch?v=khw1aXupTgk
  • WÖRLE,Franziska,«Negative Feedback Loops and Self-fulfilling Prophecies: Sociotechnical Assessment of Unfairness in Predictive...
  • WYLLIE,Sierra et al.,«Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias», 2024, pp. 1-46.
  • YUAN, Haochen et al., «Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimization», 2025,...