For years, employers have been exploring the possibility of how automated tools (ranging from basic computer screening methods to advanced artificial intelligence systems) may be able to improve and simplify the hiring process and other employment evaluation processes.  Automated tools employers use today include resume scanners that prioritize applications based on certain content; employee monitoring software that analyzes employee performance; virtual assistants and video technologies that evaluate candidates’ mannerisms and other characteristics, sometimes known as “Chatbots”; and other “job fit” algorithms programmed to evaluate the likelihood of an individual’s success in a certain position. For New York employers, most of these tools are about to be regulated unlike ever before.

Effective January 1, 2023, New York City employers will be prohibited from using artificial intelligence in employment decision-making processes unless they take a number of specific and affirmative steps prior to doing so, including a bias audit of the tool.  These requirements have emerged following the passage of New York City Local Law 144 in December 2021, which creates a specific regime employers must adhere to in order to utilize automated employment decision tools, which the City has referred to as “AEDTs”.  Specifically, employers must: (1) subject AEDTs to a bias audit within one year of its use; (2) ensure that the results of such audits are publicly available; (3) provide particular notices to job candidates regarding the employer’s use of these tools; and (4) allow candidates or employees to potentially request alternative evaluation processes as an accommodation.

Many questions emerged following the passage of Local Law 144 such as those concerning: what specifically employers must include as part of a “bias audit,” what should such an audit ascertain, and how much information must employers publicize?  Ostensibly in response to some of these inquiries, the City’s Department of Consumer and Worker Protection (“DCWP”) has proposed rules that provide some answers, expand upon Local Law 144, and regulate the use of AEDTs.  We analyze and discuss these proposed rules below.

Summary of Proposed Rules

DCWP’s proposed rules serve three primary purposes: (1) to define terms left undefined by Local Law 144 or otherwise clarify some ambiguities; (2) set forth the contours and requirements of the “bias audit;” and (3) elaborate on the notice requirements mandated by the law. 

1. Defining and Clarifying Terms

Several terms in Local Law 144 were undefined and/or ambiguous.  The proposed rules define and expand upon several terms, including the following most notable terms:  

• Independent Auditor – Defined as “a person or group that is not involved in using or developing an AEDT that is responsible for conducting a bias audit of such AEDT.”  Although this would appear to indicate that a vendor (i.e., an entity that develops and/or sells an AEDT) is not allowed to also conduct the bias audit of the tool it develops, an example in the proposed rules appear to implicitly provide that it would be permissible for a vendor to provide data for and/or perform a bias audit.  It remains to be seen whether further clarification on these seemingly contradictory messages will emerge. 

• Candidate for Employment – Defined as “a person who has applied for a specific employment position by submitting the necessary information and/or items in the format required by the employer or employment agency.”  This definition, if adopted, would confirm that a “candidate” under this law must have actually taken the affirmative step of applying for a job through the employer’s required processes. 

• AEDT – While Local Law 144 does define AEDT, the proposed rules clarify that the phrase “substantially assist or replace discretionary decision making for making employment decisions that impact natural persons”—contained within the law’s definition of AEDT—means that an AEDT: (a) relies “solely on a simplified output (score, tag, classification, ranking, etc.), with no other factors considered”; or (b) uses “a simplified output as one of a set of criteria where the output is weighted more than any other criterion in the set”; or (c) uses “a simplified output to overrule or modify conclusions derived from other factors including human decision-making.”  While we could devote an entire blog post to dissecting these additional definitions, it is worth highlighting that the proposed rules would exclude from the definition of “simplified output” “analytical tools that translate or transcribe existing text, e.g., convert a resume from a PDF or transcribe a video or audio interview. 

2. Expanding On the Contours of the “Bias Audit”

The “bias audit” was one of the more novel concepts to emerge out of Local Law 144.  The proposed rules effectively set forth the minimum requirements for a bias audit of an AEDT, which include: (1) calculating the selection rate for each race/ethnicity and sex category that is required to be reported to the EEOC pursuant to the EEO 1 Component report; and (2) calculating the “impact ratio” for each such category. 

The proposed rules define “selection rate” as the “rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification by an AEDT,” and “impact ratio” as either: (1) the selection rate for a category divided by the selection rate of the most selected category; or (2) the average score of all individuals in a category divided by the average score of individuals in the highest scoring category. 

The proposed rules also indicate that an intersectional analysis must be conducted.  That is, an analysis of the impact rate for ethnicity and sex combined, in addition to each protected category independently. 

3. Clarifying Notice and Publication Requirements

The proposed rules would require that before using an AEDT, the employer make publicly available on the career or jobs section of their website in a clear and conspicuous manner: (1) the date of the most recent bias audit and a “summary of the results” (including the selection rates and impact ratios for all categories); and (2) the distribution date of the AEDT.  Importantly, employers must keep this summary of results posted for at least 6 months after the last use of the given AEDT.

In addition to publishing a summary of the results, the law also requires that employers provide at least 10 business days’ notice to candidates and employees who reside in New York City that an AEDT will be used in connection with a given assessment or decision.  The proposed rules would allow this notice requirement to be satisfied by posting such notice on the jobs or careers section of the employer’s website, including in a job posting, or by sending to candidates for employment by U.S. mail or e-mail.  This notice must also include instructions “for how to request an alternative selection process or accommodation.”  However, the rules fail to provide any definition of “alternative selection process or accommodation,” or elaborate on the contours of what such alternatives might look like in practice.

Parting Thoughts

Employers should be aware that New York City is not the only governmental body scrutinizing these tools.  As we covered earlier this year here, the EEOC has also issued technical guidance addressing how an employer’s use of software, algorithmic decision-making tools and AI to assist in hiring workers, monitoring worker performance, determining pay or promotions, and establishing the terms and conditions of employment could violate disability protections under the Americans with Disabilities Act (“ADA”).  Further, Illinois and Maryland have already enacted laws regulating these tools to a certain extent, although not nearly as comprehensively as New York City.  Finally, the White House even recently released a comprehensive “Blueprint for an AI Bill of Rights,” which outlines several considerations germane to the use of automated decision-making tools and AI in the employment context. 

In short, employers—even those outside of New York City—that are looking to use artificial intelligence should be aware of their various applicable compliance obligations and continue to monitor for further developments on this issue.  

Employers that are already using AEDTs within New York City and other technologies in their employment decision-making processes, or plan to implement them in the workplace in the future, should consider taking the following actions:

• Review Local Law 144 and the proposed guidance to understand new compliance obligations for January 2023.

• Assess what categories of automated tools and technologies the employer uses in its workplace decision-making schemes, and determine with counsel whether this recently enacted law impacts the availability of those tools.  If so, employers should decide whether to continue using such tools (and, if so, how to ensure compliance) or otherwise whether the compliance framework may be too burdensome to continue use.

• Review the employer’s practices regarding data retention.

• Work with third-party vendors to ensure their compliance with the new law, as necessary.

• Train Human Resource professionals and other managers involved in hiring and employment decision-making processes to ensure they are familiar with the new legal requirements and to address any issues that may arise with implementation.

• If the employer is utilizing AEDTs, ensure that notices are effectively provided to any candidates or employees, where appropriate, and that the employer will be able to comply with the independent bias audit requirements.

Finally, it is worth noting that these are proposed rules, subject to public comment and a public hearing on October 24, 2022, and further consideration ahead of the law’s January effective date.  We will continue monitoring these proposed rules as they face a notice and comment period, and provide any relevant updates as the January 1, 2023 effective date nears.