How to audit an AI resume screening tool before it screens anyone

Audit an AI resume screening tool before it screens a single application. Start by checking whether the tool is covered by the laws that apply to your hiring process. In New York City, an employer or employment agency generally cannot use a covered automated employment decision tool without a bias audit, a published summary and required candidate notice.

Key facts

The audit itself should measure whether the tool produces materially different outcomes across protected groups. For NYC's required bias audit, that includes selection rates and impact ratios by sex, race and ethnicity, including intersectional categories. Then go further with your own testing. Legal compliance is the baseline, not proof that a screening system is fair or suitable for your hiring process.

The regulatory picture is also expanding. Illinois has added requirements around discriminatory effects from AI used in employment. Colorado's high risk AI framework imposes duties around algorithmic discrimination and impact assessments, although its requirements have been amended since the original 2024 legislation. The EU AI Act classifies AI systems used for recruitment and candidate evaluation as high risk under Annex III, with the applicable obligations arriving on the Act's stated implementation timetable.

Two research findings are worth keeping in mind before accepting a vendor's fairness claim. One audit found that text embedding models favoured White associated names in 85.1% of comparisons. Another study tested fourteen chat models across 24,024 paired job postings per model and found that the direction of bias changed across model generations.

The lesson is simple: do not treat an AI resume screening tool as a neutral filter until you have tested it yourself. The vendor's model card, fairness statement and compliance claims are useful evidence. They are not a substitute for an audit of the system you are actually putting into your hiring process.

In this article:

• Is AI resume screening legal in 2026, and what the four regimes actually require
• How an AI resume screener decides, from parsing to ranking
• What a bias audit actually tests, and what it does not
• What to ask a screening vendor before you buy
• What to do if you built the screener yourself
• Where this leaves your hiring process
• Conclusion
• FAQ

Is AI resume screening legal in 2026?

Yes. AI resume screening is legal in 2026, but the requirements depend on where and how the tool is used. In some jurisdictions, employers must meet specific conditions before using automated employment decision tools. Those conditions can include a bias audit, candidate notice, impact assessments or human review. The important distinction is that using AI for screening is not automatically illegal, but using it without meeting applicable requirements can be.

The four regimes, and what each one actually requires

1. New York City.

Local Law 144 has been in effect since 1 January 2023, with enforcement beginning on 5 July 2023. The DCWP says the law applies when all three elements are present:

• The tool uses machine learning, statistical modelling, data analytics or artificial intelligence.
• It assists with an employment decision.
• It substantially assists or replaces discretionary decision making.

That means not every AI tool used by a recruiter is necessarily covered. A keyword filter that a recruiter routinely overrides is different from a system that ranks candidates and substantially influences who progresses.

The DCWP also distinguishes between employment screening and other recruiting activities. Screening applicants falls within the definition of an employment decision, while searching a resume database or contacting people who have not applied does not necessarily fall within the same definition.

2. Illinois.

Illinois amended its Human Rights Act through HB 3773, also known as Public Act 103 0804. The amendments took effect on 1 January 2026. The law addresses discriminatory effects from AI used in employment decisions and specifically identifies the use of zip code as a potential proxy for protected characteristics.

Illinois also introduced notice requirements. The practical challenge is that some implementation details have required further regulatory guidance. Employers therefore need to distinguish between what the statute requires and what later guidance says about how those requirements should be implemented.

3. Colorado.

Colorado's AI law creates requirements for deployers of covered high risk AI systems used in consequential decisions. These include measures intended to prevent algorithmic discrimination. The framework also includes risk management, impact assessments and notice requirements.

One of its most important practical features is the opportunity for an affected person to correct relevant data and seek human review. That turns compliance into part of the hiring process rather than something that exists only in an audit document.

4. The European Union.

The EU AI Act classifies AI systems used for recruitment and selection as high risk when they are intended to analyse and filter job applications or evaluate candidates. The Act therefore treats automated recruitment as a higher risk use case than many ordinary business applications.

The timing is more complicated than a single effective date. The AI Act generally applies from 2 August 2026, while certain high risk provisions have later application dates. Teams operating in the EU should therefore check the specific obligation and its applicable date rather than treating 2 August 2026 as a universal deadline.

None of these regimes creates a blanket ban on AI resume screening. They create conditions around its use.

That distinction matters for employers. The question is not simply "Can we use AI to screen resumes?" It is "What does our jurisdiction require us to do before, during and after we use it?"

How does an AI resume screener actually decide?

An AI resume screener typically works through five stages:

1. It parses the resume:
2. Normalises the text;
3. Extracts features,
4. Scores the candidate;
5. And ranks the result.

The biggest problems can occur before the scoring stage. If the system fails to extract part of a resume, the final score is based on incomplete information.

The technology is also becoming easier to access. Open source projects such as easy-resume-review and recruiting-copilot show that automated screening workflows can now be built and self hosted without an enterprise recruitment platform.

Parse, normalise, score, rank

1. Parse. The system converts the resume into machine readable text. A PDF may contain selectable text, an image or a complicated layout. OCR may be needed when the document is scanned.

This is where problems can begin. Columns, tables, unusual fonts and non Latin characters can interfere with extraction. The system may not report an error. It may simply miss part of the resume.

The result is important because the next stages are working from whatever the parser managed to extract. A candidate can therefore receive a low score because the system failed to read their experience, rather than because the experience was irrelevant.

2. Normalise. The system standardises information so it can be compared. Dates may be converted into a common format. Job titles may be mapped to related terms. Names and locations may also be processed.

Regional differences can create problems here. A format the system does not recognise may be treated as missing information rather than valid information.

3. Extract features. The system identifies information it considers relevant to the job. This can include skills, experience, education, job titles and other characteristics.

This is also where proxy variables can enter. A model does not need an explicit race or gender field to produce different outcomes. Information such as location, school, employer or language can correlate with protected characteristics.

4. Score. The system assigns a score based on the information it has extracted. Some systems produce one overall match score. Others provide separate scores for different criteria.

5. Rank. Candidates are ordered according to their scores. The employer then decides where the cutoff sits or allows the system to determine who progresses.

That final cutoff is itself a policy decision. A system might rank candidates continuously, but someone still has to decide how many candidates move forward.

Why per dimension scoring is the auditable design

A screener that produces one opaque score is difficult to investigate. A screener that produces separate scores for different criteria gives you more to test.

Imagine two candidates receive match scores of 0.71 and 0.68. You know who ranked higher, but you do not know why. If the system instead reports separate scores for experience, skills, education and other criteria, you can compare those dimensions and investigate where differences originate.

This matters for bias testing. If a candidate's overall score is lower because the system penalises employment gaps, you need to know that before deciding whether the scoring rule is appropriate.

Research on text embedding models highlights the problem. A peer reviewed audit covering more than 500 resumes and 500 job descriptions across nine occupations found that document length and the frequency of names in the models' training data affected which resumes were selected.

Document length is not a measure of ability. If it influences selection, the employer needs a way to identify that effect.

This makes the tool's output an important purchasing question. Do not only ask whether a vendor claims its system is fair. Ask what information the system exposes when it makes a decision.

Proxy variables and the zip code clause

A screening model can discriminate without ever being given a protected characteristic directly.

Location is a straightforward example. In the United States, zip code can correlate with race closely enough to act as a proxy. Other resume information can work in similar ways. A school, employer, address or graduation year can contain information that correlates with protected characteristics.

Removing explicit race and gender fields therefore does not remove the possibility of discriminatory outcomes. The information can remain embedded in the free text.

The research also shows why you should not assume that every AI model will behave in the same way. Wilson and Caliskan's embedding audit found that White associated names were favoured in 85.1% of cases, while female associated names were favoured in 11.1% of cases.

A separate study tested fourteen language models across 24,024 paired job postings per model and found that the direction of measured hiring bias changed across model generations.

These studies are not necessarily contradictory. They examine different types of models and different tasks. An embedding model used to compare documents is not the same system as an instruction tuned language model evaluating a candidate.

The practical lesson is more important than the disagreement: you cannot determine how a resume screener will behave from the model name alone.

You need to test the actual system you plan to use, with the type of resumes and hiring decisions it will encounter

What does a bias audit actually test?

A bias audit primarily tests selection rates and impact ratios across demographic groups. Under New York City's Local Law 144, the required analysis covers sex, race and ethnicity, including intersectional categories. That is the legal baseline. A responsible employer should treat it as the starting point rather than the entire fairness assessment.

The DCWP describes the audit as an impartial evaluation conducted by an independent auditor. At minimum, it must calculate selection or scoring rates and impact ratios for the relevant categories.

There are three details that matter just as much as the calculations.

1. The audit expires.

The DCWP states that a bias audit must be conducted annually. An employer or employment agency can rely on an audit for one year from the date it was conducted. An audit that is eighteen months old therefore does not satisfy the annual requirement.

2. The demographics cannot simply be guessed.

The DCWP says that imputed or inferred demographic information cannot be used for the required bias audit. It also allows certain categories representing less than 2% of the data to be excluded from the calculations.

This creates a practical problem for employers. If your candidate data does not contain reliable demographic information, you may not have enough information to perform the required analysis. Guessing someone's race or gender from their name is not a substitute.

3. The audit does not require you to fix the result.

This is one of the most important limitations of the law. Local Law 144 requires the bias audit and disclosure requirements, but it does not require an employer to take a particular action when the results show a disparity.

That distinction changes how you should interpret compliance.

A tool could produce an unfavourable impact ratio, publish the required information and still satisfy the narrow requirements of the audit regime. Legal compliance therefore does not automatically mean that the screening system is fair.

The audit tells you what happened. It does not decide whether the result is acceptable. That decision remains with the employer.

What should you ask a screening vendor before you buy?

Ask the vendor six questions before buying an AI screening tool. The first is the most important: who is responsible for the bias audit? In New York City, the answer is you, not the vendor. The vendor's own audit or fairness statement does not automatically satisfy the employer's legal obligations.

Six questions to ask

1. Who is responsible for the audit?

You are. The NYC DCWP states that the vendor that created the tool is not responsible for conducting the employer's bias audit.

If a vendor says, "Our system has been audited," that does not answer the question. Ask whether the audit covers your use of the tool and whether it meets the requirements that apply to your organisation.

2. Who conducted the audit, and are they independent?

Ask for the auditor's name and their relationship with both you and the vendor.

The DCWP's guidance sets out circumstances that can undermine independence. An auditor should not work for the employer or vendor. They should not have been involved in developing or distributing the tool. They should also not have a financial interest in either organisation.

Do not assume that the word "independent" in a vendor's marketing means the audit meets the legal definition.

3. What does the tool actually output?

Ask whether the system produces one overall score or separate scores for different criteria.

A single score can tell you who was ranked higher. It gives you much less information about why. Separate dimensions make it easier to investigate whether a particular factor is driving an unwanted disparity.

This is also important when you audit the system. You want enough information to investigate a result rather than simply record it.

4. What happens to candidates who receive a low score?

Find out whether every candidate eventually receives human consideration or whether the system can automatically remove candidates from the process.

This is particularly important in Colorado, where the applicable AI framework includes an opportunity for human review in covered circumstances.

The vendor should be able to explain exactly what happens after the model produces its result. If a low score automatically prevents a candidate from reaching a human reviewer, you need to understand whether that process is compatible with the requirements that apply to your hiring process.

5. Where and when does candidate notice happen?

Ask the vendor to show you exactly how notice is delivered.

Under NYC's requirements, candidates must receive notice at least 10 business days before the employer uses a covered AEDT on them.

That makes notice part of the application workflow. It is not simply something you can add to a careers page after the system is already running.

6. Does the law apply to remote roles?

Do not assume that a remote position falls outside a local law.

The NYC DCWP specifically addresses remote work. A fully remote position can still fall within the law when the job is associated with an office in New York City.

Ask the vendor how its system handles your actual hiring locations. Then confirm the legal position independently.

The four regimes side by side

The rules are changing, and the details depend on the jurisdiction and the specific way the tool is used. Before relying on a particular compliance date or requirement, verify it against the relevant regulator or primary legislation.

The most useful vendor question is therefore not "Is your AI compliant?" It is "Show me exactly what you do, what you measure, who audited it and what happens to a candidate after the model makes its decision."

What if you built the screener yourself?

Building the screener yourself does not remove the obligations that apply to automated hiring tools. It can make compliance harder because you are responsible for the system and still need an independent audit where the law requires one.

Under NYC Local Law 144, the audit obligation applies to the employer using a covered AEDT, regardless of whether the employer built the software or bought it from a vendor. The independence requirement can also create a practical problem for small teams. Someone involved in developing the tool cannot simply audit their own work.

There are three useful steps to take before deploying a home built screener.

1. Define what the tool actually does.

Document how much influence the system has over the hiring decision.

If a recruiter reviews every application and uses the tool only to organise candidates, the system may be treated differently from one that automatically removes candidates below a particular score. The distinction matters because NYC's definition of a covered AEDT depends partly on whether the system substantially assists or replaces discretionary decision making.

2. Instrument the system before deployment.

Keep a record of the information the system receives and the decisions it produces. At minimum, that should include the original resume, parsed text, individual scoring dimensions and final ranking.

You cannot reliably audit information that you never recorded. You also cannot simply recreate missing demographic information later by guessing it from names or other characteristics. NYC's guidance does not permit imputed or inferred demographic data for the required bias audit.

3. Test the parsing layer separately.

Take a sample of resumes that represent the formats your organisation actually receives. Run them through the parser without allowing the scoring model to influence the result. Then inspect the extracted text.

Look for missing columns. Check tables. Test scanned documents. Include different fonts and layouts.

This is a simple test, but it can expose a serious problem. If the parser silently drops a candidate's experience, the scoring model may treat that candidate as less qualified even though the underlying resume contains the relevant information.

If your team cannot realistically build the logging, testing and audit process, that is useful information in itself. You may not be ready to automate screening.

A structured technical interview can be a more defensible alternative. Give candidates the same questions. Use the same scoring criteria. Record the results consistently.

The goal is not to automate hiring simply because automation is available. The goal is to make better and more defensible hiring decisions.

Where this leaves your hiring process

The compliance answer and the quality answer point in the same direction: do not let an opaque automated score make an important hiring decision without meaningful human oversight.

An automated screen sits at the top of the funnel, where there is usually the least information about a candidate. The system may be working from a formatted document. It may rely on proxy variables. It may then produce a ranking that a recruiter cannot easily explain.

The alternative is not necessarily slower hiring. It is moving important judgment to a stage where someone can explain why a candidate progressed or was rejected.

RocketDevs says it assesses developers for 6-8 hours and reports a 98%+ applicant rejection rate, with the top 2% accepted. Its Associate tier starts at $9.99/hr, with a 14-day money-back trial. That is not a compliance solution and it does not remove an employer's legal obligations. It represents a different approach to the initial screening decision: a person evaluates the candidate rather than relying entirely on an automated score. Start with a vetted developer.

A pre vetted developer pool can also move some of the screening work earlier in the process. You still need to understand how the provider evaluates candidates and what evidence it can provide. You remain responsible for your own hiring decisions.

For teams conducting the process themselves, the next stages matter just as much. You need a reliable way to evaluate technical ability. You also need a way to verify that a remote developer is who they claim to be. Those checks give you information that a resume alone cannot provide.

The broader lesson is simple. AI screening should not be treated as a shortcut around human judgment. It should only be used when you can explain what the system is doing, test whether it produces acceptable outcomes and maintain meaningful human involvement.

If you cannot answer those questions, a structured human screening process may be the safer choice. It gives you a decision that can be examined, challenged and explained. That is valuable not only for compliance, but for hiring well.

Conclusion

AI resume screening can save time, but it should never become a black box between a candidate and a hiring decision. Before using one, you need to understand what the system reads, what it measures and how it ranks people.

The legal requirements are only the starting point. An annual bias audit does not prove that a tool is fair. A vendor's fairness claim does not prove that it will behave fairly on your candidates. Removing race or gender from the input does not remove the possibility of proxy discrimination.

The safest approach is to audit before deployment. Test the parsing layer. Record the inputs and outputs. Examine selection rates across relevant groups. Understand which factors influence the score. Make sure candidates receive the required notices. Keep a human involved where the law or your own risk assessment requires it.

For smaller teams, there is also a simpler option. If you cannot properly test, monitor and govern an automated screener, do not use one yet. A structured human process with consistent questions and scoring can be more defensible than an AI system whose decisions nobody can explain.

The goal is not to automate screening. It is to make better hiring decisions without turning efficiency into a new source of legal and hiring risk.

Frequently asked questions

Is AI resume screening legal in the United States? Yes, but the requirements vary by jurisdiction. There is no general federal ban on AI resume screening, and existing employment discrimination laws still apply.

New York City has required specific measures for covered automated employment decision tools since 2023. Illinois introduced additional requirements in 2026. Colorado has also established requirements for covered high risk AI systems used in consequential decisions.

Employers therefore need to check the rules that apply to the location and type of hiring decision involved.

What is a bias audit for an AI hiring tool? Under New York City's Local Law 144, a bias audit is an impartial evaluation conducted by an independent auditor. At minimum, the audit must calculate selection or scoring rates and impact ratios across sex, race and ethnicity, including intersectional categories.

The audit must be conducted annually. Employers cannot rely on an audit that is more than one year old. Imputed or inferred demographic information cannot be used as a substitute for actual demographic data.

Does New York City's Local Law 144 apply to a company outside New York? It can. The relevant question is not simply where the company's headquarters are located.

The DCWP explains that the law can apply to fully remote positions when the job is associated with an office in New York City.

A distributed company should therefore check the location associated with each role rather than assuming that remote work removes the requirement.

Can an AI screener discriminate without seeing race or gender? Yes. A model can use other information as a proxy for protected characteristics.

Illinois specifically identifies zip code as a potential proxy in its AI employment legislation. Other resume information can also carry similar signals, including school names, employer names, addresses and language patterns.

Research has demonstrated that these effects can occur in practice. One audit of text embedding models found White associated names were favoured in 85.1% of cases. The study also found that document length could affect selection, even though resume length is not a measure of a candidate's ability.

Do I need a bias audit if I built the screening tool myself? If the tool is covered by Local Law 144, building it yourself does not remove the employer's audit obligation. The responsibility attaches to the employer using the automated employment decision tool.

Building the system yourself can make the requirement harder to satisfy because the auditor must be independent. Someone who helped develop or distribute the tool cannot simply audit their own work.

For a small team, that can make automated screening difficult to govern properly. If you cannot arrange an independent audit and the required monitoring, a structured human screening process may be the more practical option.

Sources

• New York City Department of Consumer and Worker Protection, Automated Employment Decision Tools: Frequently Asked Questions. The operative guidance on Local Law 144.
• Morgan Lewis, Illinois passes new law to address AI in the workplace, September 2024. HB 3773, Public Act 103-0804.
• Jones Day, jonesday.com, October 2024.
• Colorado General Assembly, SB24-205, Consumer Protections for Artificial Intelligence.
• EU AI Act, Annex III and Article 113, third-party mirror of the official text.
• Kyra Wilson and Aylin Caliskan, Gender, race, and intersectional bias in resume screening via language model retrieval, AIES 2024. 500+ resumes, 500+ job descriptions, nine occupations.
• Auditing hiring bias across model generations, arXiv preprint, 2026. Fourteen models, 24,024 paired postings per model.
• redmaplewww/easy-resume-review and Viy1204/recruiting-copilot, repository metadata via the GitHub REST API, 25 August 2026.
• SHRM, AI in HR: the hype and the reality. Survey of 2,040 HR professionals.

James Hitch, COO at RocketDevs. Connect on LinkedIn.