In a groundbreaking move, likely to have significant impact on employee hiring and HR tech, the New York City Council has passed a measure (“the NYC measure”) that bans the use of automated decision-making tools to (1) screen job candidates for employment, or (2) evaluate current employees for promotion, unless the tool has been subject to a “bias audit”, conducted not more than one year prior to the use of the tool.  The NYC measure will take effect January 2, 2023.

The NYC measure was passed due to growing concern about automated decision-making tools – which will also be regulated under the California Privacy Rights Act, which is set to take effect at the same time as the NYC measure – one of which is that such tools may be imbedded with unintended biases that result in outcomes that discriminate against individuals based on protected characteristics like race, age, religion, sex and natural origin.

The category of automated decision-making tools targeted by the NYC measure is “automated employment decision tools,” which the measure defines as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”  Excluded from the measure’s scope are tools that do not automate, support, substantially assist or replace discretionary decision-making processes and that do not materially impact natural persons, such as, for instance, junk email filters, firewalls, antivirus software, calculators, spreadsheets, databases, data sets, or other compilations of data.

Employers that intend to utilize an employment decision tool must first conduct a bias audit and must publish a summary of the results of that audit on their websites.  They must also notify all NYC employees and/or job candidates that: (1) the tool will be used in connection with assessment or evaluation of their employment or candidacy and (2) specify the job qualifications and characteristics that the tool will use to make that assessment or evaluation.

Utilizing an automated employment decision tool without first conducting a compliant bias audit exposes employers to civil penalties of up to $500 on day one, followed by penalties of $500 to $1,500 every day thereafter.   Failure to properly notify candidates or employees about use of such tools constitutes a separate violation.

This is not the first legislation of its kind, but certainly the most expansive.   In late 2019, Illinois passed the Artificial Intelligence Video Interview Act (“the AIVI Act”), HB2557, which imposes consent, transparency and data destruction requirements on employers that implement AI technology during the job interview process. The AIVI Act, the first state law to regulate AI use in video interviews, took effect January 1, 2020. Likewise, in 2020, Maryland enacted a law that requires notice and consent prior to use of facial recognition technology during a job interview.  And the Attorney General of Washington D.C. recently introduced a bill that addresses discrimination in automated decision-making tools generally.    Similar legislation is likely to trend across other states, as this technology continues to infiltrate hiring practices and other areas of business.  As early as 2014, the EEOC has been taking notice of “big data” technologies and the potential that the use of such technology may be in violation of existing employment laws such as Title VII of the Civil Rights Act of 1964, the Age Discrimination in Employment Act, the American with Disabilities Act and the Genetic Information Nondiscrimination Act.

Only time will tell the impact the NYC measure and others of its kind will have on employment practices, but employers should tread carefully with AI usage in the workplace. Moreover, it will likely not be long before other states and localities enact similar legislation. Employers, regardless of jurisdiction, should be evaluating their hiring practices and procedures, particularly to ensure that they obtain appropriate written consent before using any technology that collects sensitive information about job applicants or employees, and that they have conducted all requisite privacy and bias impact assessments.