Key Takeaways
- AI is already in your hiring process. 58% of attendees are piloting AI or using approved tools in their workflows. 40% said their biggest AI hiring compliance worry is simply figuring out which rules apply.
- Federal law is still the bedrock. Title VII, the ADEA, and the ADA cover AI tools. Plaintiffs can bring disparate impact claims even as federal agencies step back from them, and cases like Mobley v. Workday are testing vendor liability.
- State laws are stacking up fast. Colorado’s automated decision-making law takes effect January 1, 2027, and requires adverse-decision notices plus human review. That can shift recruiter savings into compliance headcount.
- “Efficient” doesn’t mean “effective.” In one real-world efficacy test, 80% of candidates who received offers had scored “Fair” or “Low” on the AI match tool.
- Validate against real outcomes. One match tool’s “Strong” scores predicted two-year retention (88% vs. about 75%) but not higher performance ratings.
- The 80% rule isn’t enough. A tool can pass the four-fifths rule and still show a statistically significant disparity. Ask for both tests.
End of summary box.
AI in Hiring Is Already Here. Part 1 of Our Playbook Recap Asks Whether It’s Actually Working.
Four Plays, Four Hours, and One Honest Question: Does Your AI Actually Work?
There’s a particular kind of quiet that settles over a virtual room when a statistician puts a table on screen and says, “This is a real-world example. This is what we see all the time. This is not an anomaly.” Our recent AI in Talent Acquisition Playbook: Hits, Misses & Compliance Strategies had a few of those moments.
Hosted by DirectEmployers on October 14, 2026, the half-day virtual event brought HR leaders, legal counsel, and talent operations pros together for four hours built around a simple premise. Jaime Costilow, VP of Marketing at DirectEmployers, framed it in her welcome: AI “brings real opportunity, along with real questions about risk, compliance, and whether the tools are actually delivering. Today is about really cutting through all the noise.”
This is Part 1 of our two-part recap, covering AI hiring compliance and how to measure whether your tools actually work. Part 2 covers PJ Rossi’s AI readiness framework and the top hits and misses from the field.
The lineup made that possible:
- PJ Rossi, MS, SPHR, of Forté Advisory Group, the self-described “data geek” who brought the practitioner’s view from years in corporate TA and HR leadership
- Michelle Duncan, Esq., Partner at Silberman Law, who has practiced employment law for nearly three decades and spends roughly 75% of her time helping employers evaluate and vet AI tools
- Krystal Welland, Lead Statistician at Silberman Law, who turned abstract talk about “bias audits” into real numbers
PJ set the tone right away. “I think we are in one of the most exciting and equally terrifying times in our professional lives,” she said. “When that work involves people, employment decisions, data, sometimes sensitive information, the consequences of a major mistake feel very, very real.”
Where the Room Stood on AI Adoption
The first audience poll asked a straightforward question: Where is your organization today with AI in talent acquisition?
- 32% are piloting selected uses
- 26% are using approved tools in regular workflows
- 21% are exploring, with no active pilot
- 13% aren’t sure
- 8% are using AI informally without a consistent approval process
In other words, close to six in ten attendees already have AI touching their hiring process in some form. “AI isn’t coming. It’s already here,” Michelle said. She pointed to SHRM research from March 2026 showing 62% of organizations use AI somewhere in their business, 43% use it in HR, and 27% use it in recruiting. And 89% of HR users report greater efficiency.
“What that tells me is that overwhelmingly, people who are using AI love what they’re seeing,” Michelle said. “We’re going to talk through whether they’re seeing the whole picture.”
That question, are we seeing the whole picture?, became the thread running through the whole day, starting with the law and the data.
Play 1: The Legal Landscape & Exposure Points, or “Don’t Let Analysis Paralysis Win”
Michelle opened the legal session with reassurance. “Do not be intimidated if you are not a techie,” she said. “The tools that we have in our toolbox can absolutely help us navigate this landscape just as it always has in the past.”
The audience clearly needed to hear that. When asked which area of AI hiring compliance concerns you most?, the room answered:
- 40%: Determining which requirements apply
- 21%: State AI employment requirements
- 19%: Federal employment discrimination requirements
- 15%: Privacy and candidate information
- 4%: Local requirements, including NYC
The biggest worry wasn’t any single law. It was the patchwork itself.
The Compliance Pyramid: Federal Law is Still the Bedrock
Michelle’s answer was a framework she called the compliance pyramid. At the base sit the federal non-discrimination laws HR teams already know: Title VII, the Age Discrimination in Employment Act (ADEA), and the Americans with Disabilities Act (ADA). State laws stack on top of those, and local laws sit at the peak. All three layers apply at the same time.
Federal non-discrimination laws form the foundation, with applicable state and local requirements adding additional layers.
None of these federal laws mention artificial intelligence, and they don’t need to. Title VII has long allowed challenges to facially neutral selection tools that disproportionately affect a protected group. That’s the disparate impact theory. Michelle used a warehouse lifting test as the classic example: the test may screen out more women, and whether that’s lawful turns on whether the employer can prove it is “job-related and consistent with business necessity.”
And while federal agencies have stepped back from disparate impact enforcement, Michelle was blunt about what that does and doesn’t change. “It’s baked into existing federal non-discrimination law. And unless Congress goes in and amends Title VII, that is a framework that plaintiffs will continue to use,” she said. “The EEOCs of the world may not use disparate impact as a theory of discrimination. But plaintiffs can.”
Don’t expect a single federal AI-in-hiring law to sweep the patchwork away, either. PJ’s take on the odds: “My crystal ball, or my Magic 8 Ball, really stopped working a really long time ago.”
The state and local patchwork
“Could the states have all gotten together and said, hey, let’s agree on some terminology here? They could have. Did they? No,” Michelle said. Her tour of the landscape included:
- Illinois: the AI Video Interview Act (triggered when AI scores a video interview) and an amended Human Rights Act that flags zip code as a proxy for race
- California: amended FEHA regulations on algorithmic discrimination and record-keeping, plus the new “No Robo Bosses” law
- New Jersey: amended its non-discrimination law to address “algorithmic bias” and codify disparate impact
- Colorado and Connecticut: broad “automated decision-making” laws centered on notice, transparency, and (for the first time) obligations on the developers of AI tools
- New York City: Local Law 144, which requires an independent bias audit and notice before using an automated employment decision tool (AEDT)
Two details stood out. First, NYC’s bias audit only requires an impact ratio analysis (the “80% rule”). It requires no statistical significance testing and gives no guidance on job-relatedness. “Here’s where they kind of missed the boat,” Michelle said, foreshadowing Play 2.
Second, Colorado’s new law, SB26-189, which takes effect January 1, 2027, requires employers to give unsuccessful applicants something close to a Fair Credit Reporting Act-style notice of an adverse decision, along with the chance to correct their data and request human review. That reshapes the ROI math. “You are saving on the front end by using an AI tool,” Michelle said. “But now you need to think about how those resources are being shifted.” The law applies to employers doing business in Colorado, not just those headquartered there.
PJ backed that up from the practitioner’s chair. When organizations tell her headcount reduction is the primary goal, she tells them that’s not realistic, because that headcount “just needs to move to go to the monitoring and the assessing of the outputs.”
What to Ask Vendors, “Beyond the Glossy Marketing Material”
PJ shared a story every TA leader should hear. A CHRO called her after discovering that a boxed AI tool was quietly excluding candidates with employment gaps or long years of service. “I’m sure all your spidey senses are going off right now,” she said, “because both of those violate our employment laws on a regular basis.”
Her bottom line: “I need to see the decision logic. I need to see the model that’s running in the background in order for me to validate it if it’s fit for purpose in my organization.”
Michelle’s baseline due-diligence list for any AI vendor:
- The intended purpose of the tool (recruiting, selection, L&D), which determines which laws apply
- A description of how the tool works, written so it could be shared publicly
- The inputs (what data and criteria are evaluated) and the outputs
The Takeaway: Find Your Lowest Common Denominator
Rather than freezing in what she called “analysis paralysis,” Michelle urged employers to build a baseline compliance framework on federal non-discrimination principles and the common themes regulators keep returning to: disproportionate impact, transparency and explainability, job-related data, intended vs. actual use, and ongoing monitoring. Then layer specific state and local requirements on top.
She also offered a reminder for employers outside NYC. In California and Connecticut, evidence of anti-bias testing is relevant both to a discrimination claim and to its defense. “It might be money really well spent to do that kind of an assessment,” she said, “because that is what plaintiffs will use, and that is what you will use as an employer to defend against a possible claim.”
Play 2: Measuring Efficacy & ROI, Where the Data Got Uncomfortable
Play 2 opened with a question from the chat about Mobley v. Workday, the case working its way through federal court in California. Instead of suing an employer, plaintiffs sued the ATS/HRIS vendor, alleging that AI screening features discriminated based on race, age, and disability. The court rejected Workday’s argument that it couldn’t be liable because it wasn’t the employer, finding it could stand in the employer’s shoes as an agent. “We think this is really just the tip of the iceberg,” Michelle said.
That’s why a real bias audit, in Silberman Law’s view, asks two questions, not one:
- Efficacy: Does the tool actually work?
- Fairness: Does it produce disparities that could be problematic under non-discrimination law?
Start with the “Why”
Krystal’s first rule: define success before you measure it. “Once we’ve identified the business problem, or really the business objective, we want to translate it into something that we can actually measure,” she said. Efficiency might mean fewer recruiter hours reviewing applications. Better hiring might mean improved performance or retention.
Then zoom out. “Improving one part of the process doesn’t necessarily mean that we’ve improved the process overall,” Krystal said. If AI cuts screening time but hiring managers now interview twice as many people to find one fit, what was really saved?
PJ added a cost line most ROI models miss: “Measure the review and the rework as part of your costs.” She also named the elephant in the room: budget pressure. “I’ve sat in that seat,” she said. “PJ, I need a $1.2 million save by the end of the year. What are you going to do about it?”
Michelle’s counter-argument for leadership: “If you have tools that will help you identify who will stay with your organization, the ROI is quantifiable.” Turnover has a real price tag, and it can dwarf recruiter salary savings.
The Efficacy Test: AI vs. Human Decisions
Then came the data. Krystal walked through a real (anonymized) employer that ran a year of historical applicants through an AI skills-matching tool in a sandbox environment, before the tool ever made a live decision. Here’s what they found:
- 85% of candidates human recruiters advanced to interview scored only “Fair” or “Low” on the AI match score
- 80% of candidates who received offers scored “Fair” or “Low”
- 50% of candidates recruiters marked “Does not meet basic qualifications” scored “Strong” or “Good”
- 60% of candidates reviewed and not advanced scored “Strong” or “Good”
In one employer’s historical-data test, most candidates who advanced to interview or received offers had received Fair or Low AI match scores.
“This is a real-world example. This is what we see all the time. This is not an anomaly,” Krystal said.
Michelle was quick to note that human decisions aren’t a perfect benchmark, and the misalignment isn’t always the vendor’s fault. “Sometimes the reasons why these results are what they are is because our job descriptions are so poorly written,” she said. Or the tool may be reading only application fields and ignoring attached résumés. Both are fixable. “But if you don’t know that there’s misalignment, if you don’t do this type of analysis, you will not identify this as a red flag.”
Following Hires Through Performance and Retention
The more advanced analysis follows hires after they start, which takes at least one performance cycle (or two years, for retention) of data:
- Performance: “Low” match scores correlated with the lowest average rating (3.2). But “Strong,” “Good,” and “Fair” clustered together (3.8, 4.0, 4.2), with “Strong” actually second-lowest.
- Retention: “Strong” matches had an 88% two-year retention rate. Everyone else hovered around 75%, no matter their score.
The practical lesson: a tool may be valid for one narrow use. You might screen out “Lows” to predict performance, or prioritize “Strongs” to predict retention. Without the data, you’d never know which. Michelle’s advice is to ask vendors directly whether they’ve done a validity study, and to treat claims that a tool is “bias-free” and “bulletproof” as something to verify, not accept.
Fairness: the 80% Rule Isn’t the Whole Story
Krystal closed with what she called a pet peeve: bias audits that stop at the four-fifths rule. In her gender example, women landed in the top match categories at 3.6% versus 4.4% for men. That’s an impact ratio of 82.3%, which passes the 80% rule and is where NYC-style audits and most vendor audits stop.
But with large applicant volumes, small gaps become statistically significant. The same data showed a standard deviation of 2.69, well past the 1.96 threshold. “Big numbers are bad numbers,” Krystal said, quoting colleague Mickey Silberman. “If we don’t run that standard deviation test, we don’t even know about that statistical significance that exists in our data.”
Why it matters legally, per Michelle: “The burden shifting occurs when we fail the standard deviation test.” Once that happens, the employer has to justify the tool. Her gateway question for vendors and auditors is simple: will you run both tests?
Krystal summed up the whole Play in one line: “Just because the tool is effective, it doesn’t mean it’s fair.”
Compliance Asks If It’s Legal. The Next Question Is Whether It Works.
The first half of the Playbook made one thing clear. The tools HR and TA teams already have still apply: non-discrimination principles, job-relatedness, and documentation. AI doesn’t replace that expertise. It raises the stakes for using it.
Build your AI hiring compliance baseline on federal law and layer state and local requirements on top. Test every tool for efficacy and fairness, using both the 80% rule and statistical significance. And don’t take “bias-free” on a vendor’s word.
Continue With Part 2
Knowing the risks is half the battle. In Part 2 of our recap, PJ Rossi walks through her nine-check AI readiness framework, including the meeting where “literally, everybody pointed at each other.” Then she and Michelle trade real-world hits and misses, from shadow AI to a benefits bot that actually works.
Keep the Conversation Going at DEAMcon27
PJ Rossi, Michelle Duncan, and Krystal Welland will all be on-site at DirectEmployers 2027 Annual Meeting & Conference (DEAMcon27), April 21–23, 2027, in Newport, Rhode Island.
Michelle didn’t hold back when asked about it. “DEAMcon is one of the best conferences I have ever attended,” she said. “If you only have the ability to choose one conference each year, I would put DEAMcon really high on that list.”
In the meantime, access the event replay in DE Connect, along with the companion summary resource. Keep an eye out for Member office hours to ensure you have the knowledge and information you need to excel in your daily responsibilities!
THIS COLUMN IS MEANT TO ASSIST IN A GENERAL UNDERSTANDING OF THE CURRENT LAW AND PRACTICE RELATING TO OFCCP. IT IS NOT TO BE REGARDED AS LEGAL ADVICE. COMPANIES OR INDIVIDUALS WITH PARTICULAR QUESTIONS SHOULD SEEK ADVICE OF COUNSEL.
Let’s chat!
Fill out this form to indicate which solutions you’re interested in and we'll contact you shortly to schedule a demo.