New technology has a way of amplifying existing human issues. It’s up to HR leaders to address those problems before AI is introduced and creates legal risk.

By Gillian Manning

In the HR sphere, AI is often described as a sort of shortcut—a faster way to screen candidates and breakdown workforce decisions. But, according to Marcus Holt, a people and culture analyst at the utilities company Salt River Project, this popular framing is fundamentally flawed.

“AI in HR is not a shortcut, it’s a responsibility. When we introduce automated solutions, the responsibility doesn’t disappear. It relocates upstream, starting well prior to implementation.”

The ongoing process of deciphering how to integrate AI has created a complicated situation for industry leaders. While there is a strong desire to build inclusive systems, the legal landscape is largely unsettled. To navigate this uncertainty, Holt says that organizations must reframe their approaches entirely around people, processes, and tools before “pouring gasoline on the fire” by scaling already-flawed processes with automation.

Grounding the Legal Landscape

Organizations may fear that putting equity at the forefront creates legal risk under Title VII of the Civil Rights Act of 1964, which prohibits discrimination based on traits like race and sex in core employment practices. However, Holt says that avoiding visibility is a dangerous mistake.

“Under Title VII, employers are allowed to examine aggregate demographic data to understand whether systems are producing unjustified disparities,” Holt says. “In fact, that awareness is expected from a legal standpoint. Avoiding visibility doesn’t eliminate bias; it just makes it harder to detect and correct. Automated tools scale embedded assumptions.”

If an HR system reveals automated disparities, Holt recommends checking the U.S. Equal Employment Opportunity Commission’s (EEOC) four-fifths (80%) rule to benchmark adverse impact; seeking outside counsel; and evaluating if the issue stems from the tool or operational inputs (like job descriptions). It’s important to note: 2023 EEOC guidelines clarify that employers cannot outsource accountability to external vendors.

“It is still your responsibility to own the tools you use,” Holt says.

Redefining Accuracy

The primary selling point for AI is technical accuracy, but Holt notes that technical accuracy doesn’t necessarily mean objective fairness.

“We have to shift the definition of accuracy in this conversation,” Holt says. “AI confirms and does exactly what we tell it to do; it isn’t making objective decisions for us. The model itself is often not broken—it is highly accurate to the historical outcomes and inputs provided.”

This becomes problematic because of “proxy variables”—context clues that correlate with protected characteristics. Holt shares a real-world case of an automated resume screening tool that heavily filtered out female applicants. The system wasn’t explicitly told to exclude women, but it decided that on its own based on a historical dataset of past male hires, treating context clues like “captain of women’s soccer team” or references to female-based schools as negative indicators. Similarly, Holt shared a study based in Cape Town, South Africa, which showed that utilizing neutral metrics like zip codes can inadvertently categorize individuals by ethnicity by proxy.

“It’s a clear example of how we design processes that look great to us, but the perspective on the other end is vastly different,” Holt warns. “It’s like parents designing a baby’s crib mobile. The parents look down from the top and think it looks amazing, but from the baby’s perspective looking up, they are just staring at a bunch of animal butts. Unless we bring in other perspectives, we will accelerate programs using AI and think they are getting better, while the experience on the receiving end is disconnected.”

Beyond Human-In-The-Loop

To combat these biases, “human-in-the-loop” systems—in other words, automation with some level of human oversight—can’t just be for show.

“If a human’s presence is just ornamental, it shifts blame to an individual contributor without producing good results,” Holt says. “Responsible design means empowering that person to speak up.”

When an organization invests resources into an expensive AI solution, a lower-level employee may find it difficult to challenge the tool. “These conversations must happen prior to onboarding,” Holt says. “If we don’t start with human-based processes and a speak-up culture, we are just accelerating errors. These systems won’t own the outcomes; we do.”