Who is Hiring You? Gender bias in AI-driven recruitment and the regulatory gaps after the AI Act.

📢 New EUFutures Blog Section! 

We just launched our new blog section where we highlight student research bridging academia and public policy. We start with a piece on "Who is Hiring You? Gender bias in AI-driven recruitment and the regulatory gaps after the AI Act," showcasing the work of our Public Affairs students Annachiara Esposito, Sonat Ezgi Alper, and Roger Figols Riera. Developed for Chiara Benassi’s course "The Political Economy of Labour Markets," this article looks at how recruitment algorithms reproduce gender biases and where the EU AI Act still falls short.

Enjoy!

___

Imagine applying for a job you are fully qualified for and never hearing back, not because someone else is better skilled, but because an algorithm decided that you did not fit. No explanation. No appeal. No human control involved. This scenario is not so far from the reality for jobseekers across Europe and beyond.

According to the 2024 European Working Conditions Survey, 42.3% of EU workers are now affected by algorithmic management (AM) and 74% of recruiters use AI in at least one of the stages of the selection process, from screening applications to interviewing and making the final selection. As AI is reshaping the recruitment process across Europe, even if at different speeds, the EU recognized the risks and intervened to regulate it.

The AI Act (Regulation EU 2024/1689) classifies AI systems used in recruitment as high-risk, mandating human oversight, transparency, and monitoring to avoid bias. Moreover, the Quality Jobs Roadmap, launched in December 2025, promises to align EU employment policies with the technological developments of the labor market, addressing the concern of 84% of Europeans that believe these technologies must be carefully managed at work. However, the question remains whether the AI Act and the scope of the roadmap meet the urgency of the challenge.

This post addresses this question and discusses what would need to change for algorithmic fairness in recruitment to move from a principle on paper to practice in the workplace.

Algorithm Discrimination

The expansion of AI-driven tools is not an impartial development. Specifically in the field of labor recruitment, AI constitutes one of the areas where use without human oversight raises crucial questions around fair and equal treatment. Some of the main reasons include algorithm training and representational bias.

The biases arise from the absence of proportionality in the data and from inappropriate usage, meaning that the system's users — in this case, companies — hold different values from those assumed during the algorithm's design process.

Algorithmic bias can also stem from programmer bias or from the data used to train the AI system. If the system is trained on historical employment data that reflects implicit bias favouring one group over another, then, without ever seeing gender or ethnicity as inputs, it can learn proxy features correlated with those characteristics and use them to disadvantage applicants, creating systemic discrimination. Research that when genders are balanced in the candidate slate, gender bias in many professions with skewed workforce distributions can be mitigated. During the initial screening of candidates, the shortlists generated through AI tools are the least gender-fair, reproducing structural inequalities that undermine the fairness of selection outcomes. Algorithms can discriminate directly — that is, they can be responsible for 'unfavourable treatment' or 'differential treatment', such as automatically discarding a female applicant's CV — but they can also produce more subtle indirect gender effects, for instance by perpetuating gender stereotypes in the labour market. Furthermore, a European Commission report highlights the opacity of advanced algorithms, which makes it difficult for companies to understand how recruitment algorithms are trained and how they operate, and for individuals to discover whether and how they have been discriminated against. Because traditional EU non-discrimination law relies on comparative frameworks and, in particular, on proof of less favourable treatment, algorithmic complexity conceals bias behind correlations and proxies.

For this reason, the EU introduced the first legal framework governing the use of AI in situations considered to present significant risk, including AI use in recruitment and selection. The framework aims to foster trust between European citizens and AI systems, while mitigating the potential risks of their use. To this end, it establishes four risk tiers into which AI systems are placed based on their functions and potential for harm, each carrying its own regulations and compliance obligations that must be satisfied before the systems can be deployed on the market. But is it enough?

The shortcomings

We argue that the roadmap should extend its focus to labour market candidates in the recruitment phase. Workers who are never hired cannot access collective bargaining protections, cannot rely on workplace representatives, and cannot invoke employment-related rights that presuppose an existing employment relationship. As a result, they are filtered out before the protective framework even begins to apply, and can only seek redress on an individual basis.

Enforcement mechanisms applicable to this phase remain underspecified. The roadmap concedes that enforcement remains uneven across Member States but proposes no mechanism to address this specifically for AI recruitment. It briefly refers to "EU rules on worker information and consultation", which "require involving workers' representatives on decisions likely to lead to substantial changes in work organisations, including deployment of AM systems" — but trade unions and works councils have hardly any say in the recruitment process.

From August 2026, the AI Act became more specific regarding its enforcement mechanisms, primarily by activating transparency obligations. These require employers to notify candidates when AI has been used in the recruitment process, and give rejected candidates the right to request an explanation of the role AI played in their assessment. If candidates believe they have been treated unfairly, they may lodge a complaint with their national market surveillance authority, which can impose penalties on companies of up to €35,000,000. Separately, high-risk AI obligations — including mandatory human oversight, monitoring, and the suspension of systems if issues arise — are scheduled to apply to employment-related AI from December 2027 under the revised implementation timeline. However, the Act's enforcement trajectory remains uncertain.

Full compliance for high-risk AI systems was required from August 2026 but has now been postponed to December 2027; in particular, the provisions concerning the data used to train and test models. This data must be relevant, sufficiently representative, and controlled to limit errors and bias. Companies must also explain where it comes from, how it was collected, what changes it has undergone, and what imbalances it contains.

The tension with the GDPR is also real and unresolved: the principles of data minimisation and purpose limitation sit in structural conflict with Article 10(5)'s permission to process sensitive data for bias detection. Legal analysis of the overlaps between the AI Act and the GDPR confirms that the two frameworks must be applied in parallel, with the GDPR taking precedence as lex specialis, but clear, accessible guidance on when the Article 10(5) exception applies in employment contexts has not materialised.

A further enforcement gap concerns the question of liability. When an AI system discriminates in recruitment, who is responsible? Blurred lines of accountability are a serious issue because corporate self-regulation has historically proven insufficient to prevent discriminatory outcomes, and AI vendors frequently market their tools as objective despite mounting evidence of systemic bias.

Theoretically, the AI Act divides responsibility between the provider and the deployer; however, it makes no provision for compensating affected individuals. Neither the employer nor the algorithm developer is under an obligation to provide direct compensation to individuals who have suffered discrimination. Redress is only possible where specific national or EU legislation establishes liability for certain forms of discrimination — but the AI Act itself makes no such provision.

Recommendations

The EU cannot dictate how private companies design their algorithms, but it can set requirements for what those algorithms must demonstrate and determine the consequences when they fail to do so. Several directions deserve serious consideration in the forthcoming Quality Jobs Act.

The most urgent step is to extend anti-discrimination obligations explicitly to the pre-hiring phase. The AI Act's high-risk classification is necessary but not sufficient: it needs procedural grip specifically in the recruitment context, including candidate notification, meaningful explainability requirements, and accessible redress mechanisms for rejected applicants.

On the preventive side, public authorities should develop guidance and training resources for developers and deployers on their obligations under anti-discrimination law and how these apply to AI-based recruitment tools.

A further priority is the democratisation of algorithmic oversight — for example, through mandatory third-party bias audits of AI recruitment tools, conducted by bodies independent of both vendors and deployers. Under the AI Act, developers can demonstrate compliance by adhering to standards developed by standard-setting organisations (SSOs); however, these standards are typically drafted by industry experts, with civil society groups and consumer organisations playing only a marginal role. In the process of deciding what can be considered 'unfair' or 'unbiased', larger firms can intervene with incentives to steer the outcome. Responsible AI think tanks and civil society organisations have been calling for this kind of structural accountability, but without legislative backing, their role remains advisory rather than corrective. Without independent oversight, detecting potentially harmful algorithms becomes more difficult, as companies grading their own homework have no incentive to surface uncomfortable findings. Moreover, diversity competence should be embedded in the development of AI systems and accompanied by sufficient resources (Article 77 of the AI Act) as a precondition for deployment in high-risk employment contexts, not as an optional ethical commitment.

Finally, the GDPR–AI Act tension requires active regulatory guidance rather than mere references to both frameworks and an assumption of self-harmonisation. Clear analysis of how Article 10(5) interacts with GDPR obligations illustrates the complexity of applying both frameworks simultaneously in employment contexts. Supervisory authorities need to issue practical, accessible guidance on when and how the Article 10(5) exception applies — guidance that would significantly reduce the legal uncertainty that currently deters good-faith bias correction efforts.

Conclusion

The analysis of the legislative framework revealed that the AI Act recognises algorithmic recruitment as a clear risk, and the Quality Jobs Roadmap signals the need to address technological advancements in the workplace; however, the framework falls short of the structural measures needed to address these risks.

We identified three shortcomings: anti-discrimination protections are mainly designed to protect people once they are in an employment relationship, so labour market candidates do not have access to legal protection before being hired; enforcement mechanisms are underspecified and are likely to be implemented unevenly, leaving outcomes dependent on each Member State's institutional capacity; and accountability has been left undefined, with no clarity on vendor versus deployer liability.

For these reasons, we identified recommendations to address these critical shortcomings: extending anti-discrimination obligations ex ante into the pre-employment phase; mandatory third-party bias audits to counteract the lack of incentives inherent in self-regulation; and clear guidance on Article 10(5)'s relationship with the GDPR. The Quality Jobs Act announced in the roadmap's final section offers an opportunity to translate aspiration into binding commitment — addressing AI discrimination through specific mechanisms embedded in industrial relations and labour market institutions, and making algorithmic fairness more than a principle: a practice.

---

References

European Commission. 2025. Quality Jobs Roadmap. Brussels: European Commission. https://commission.europa.eu/document/download/e6cd4328-673c-4e7b-a89e-7a38b6a7c7d5_en.

European Commission. 2020. "Commission Sets Out Roadmap for the Future of Social Protection." Employment, Social Affairs & Inclusion. https://employment-social-affairs.ec.europa.eu/news/commission-sets-out-roadmap-future-2020-01-14_en.

European Parliament. 2025. The AI Act and the GDPR: Data Governance, Overlaps, and Synergies in High-Risk AI Systems. European Parliamentary Research Service (EPRS) Study. PE 774.670. Brussels: European Parliament. https://www.europarl.europa.eu/RegData/etudes/STUD/2025/774670/EPRS_STU(2025)774670_EN.pdf

European Union. 2024. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain legislative acts. Official Journal of the European Union, L 2024/1689, July 12, 2024. http://data.europa.eu/eli/reg/2024/1689/oj

Gerards, Janneke, and Raphaële Xenidis. 2021. Algorithmic Discrimination in Europe: Challenges and Opportunities for Gender Equality and Non-Discrimination Law. European Equality Law Network (EELN). Brussels: European Commission. https://ai.equineteurope.org/system/files/2021-04/EELN_Algorithmic-discrimination_GERARDS%20and%20XENIDIS_2021.pdf.

Haque, Adnan. 2025. "AI is Hiring You: Algorithmic Power, Ethical Risks, and the Future of Recruitment." International Journal of Organization Theory & Behavior. Ahead-of-print. https://doi.org/10.1108/IJOTB-02-2025-0045.

Hartmann, David, José Renato Laranjeira de Pereira, Chiara Streitbörger, and Bettina Berendt. 2025. "Addressing the Regulatory Gap: Moving Towards an EU AI Audit Ecosystem Beyond the AI Act by Including Civil Society." https://link.springer.com/article/10.1007/s43681-024-00595-3 

Holtz, Hajo Michael, and Jonas Ledendal. 2026. "AI Data Governance – Overlaps between the AI Act and the GDPR." Law, Innovation and Technology 18, no. 1: 1–25. https://doi.org/10.1080/17579961.2026.2633677.

Kaya, Mesut, and Toine Bogers. 2026. "Human, Algorithm, or Both? Gender Bias in Human-Augmented Recruiting." In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT '26), 5309–5326. New York: ACM. https://doi.org/10.1145/3797143.3797611.

Kieslich, Kimon, et al. 2026. "Trade-Offs in Deploying Legal AI: Insights from a Public Opinion Study to Guide AI Risk Management." https://arxiv.org/pdf/2603.06240.

Köchling, Alina, and Marius Claus Wehner. 2020. "Discriminated by an Algorithm: A Systematic Review of Discrimination and Fairness by Algorithmic Decision-Making in the Context of HR Recruitment and HR Development." Business Research 13, no. 3: 795–848. https://doi.org/10.1007/s40685-020-00134-w.

La Repubblica. 2026. "AI Act 2 agosto 2026: norme, obblighi, rinvii." la Repubblica, July 31, 2026. https://www.repubblica.it/tecnologia/2026/07/31/news/ai-act-2-agosto-2026-norme-obblighi-rinvii-425504792/.

Lütz, Fabian. 2022. "Gender Equality and Artificial Intelligence in Europe. Addressing Direct and Indirect Impacts of Algorithms on Gender-Based Discrimination." ERA Forum 23, no. 1: 33–52. https://doi.org/10.1007/s12027-022-00709-6.

Mariniello, Mario. "The Right Balance: How to Fix European Union Artificial Intelligence Regulation." Policy Brief 12/2026, Bruegel, 11 giugno 2026. https://www.bruegel.org/policy-brief/right-balance-how-fix-european-union-artificial-intelligence-regulation.

Njoto, Sheilla, Marc Cheong, Reeva Lederman, Aidan McLoughney, Leah Ruppanner, and Anthony Wirth. 2022. "Gender Bias in AI Recruitment Systems: A Sociological-and Data Science-based Case Study." In Proceedings of the 2022 IEEE International Symposium on Technology and Society (ISTAS), 1–7. https://doi.org/10.1109/ISTAS55053.2022.10227106.

Peng, Andi, Besmira Nushi, Emre Kıcıman, Kori Inkpen, Siddharth Suri, e Ece Kamar. 2019. "What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring." Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, no. 1: 125–34. https://doi.org/10.1609/hcomp.v7i1.5281.

Themann, Sarah. 2025. "Challenges and Strategies Used In Implementing AI Governance: A Systematic Literature Review." Master's thesis, Uppsala University. https://doi.org/10.1007/s43681-024-00595-3.

Wang, Tianshu, Peng Huang, and Randal Burns. 2023. "Understanding and Dealing with Hard Faults in Persistent Memory Systems." In Proceedings of the 2023 IEEE International Conference on Computer Design (ICCD), 441–457. Piscataway, NJ: IEEE. https://ieeexplore.ieee.org/document/10227106.

Wuidar, Simon, Laura Engels, Olivier Lisein, Patrizia Zanoni, Giseline Rondeaux, and Thomas Pirsoul. 2026. Recruitment and Selection in the Age of AI: AI Use, Awareness of Bias Risks and Recommendations Regarding the Risks of Gender Discrimination – Policy Brief. Brussels: Institute for the Equality of Women and Men. https://hdl.handle.net/2268/340904.