On September 23, 2022, the New York City Department of Consumer and Worker Protection published proposed rules to implement the city’s automated employment decision tools (AEDT) law. The law, which will take effect on January 1, 2023, conditions the use of automated employment decision tools by employers and employment agencies on their compliance with certain requirements, including the performance of bias audits and the furnishing of notifications to candidates and employees. The proposed rules define several key terms, identify the requirements for a bias audit, address obligations for publishing the results of a bias audit, and specify the notices to be furnished to employees and candidates for employment.

Clarification of the Automated Decision Tools Triggering Applicability of the Law

The law specifies that it is applicable to automated employment decision tools, which the law defines as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation” that is used to “substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.” Although the law states that it does not apply to a tool that “does not automate, support, substantially assist or replace discretionary decision-making processes and that does not materially impact natural persons,” until now, guidance did not exist about the meaning of the phrases “machine learning, statistical modeling, data analytics, or artificial intelligence,” “simplified output,” or “substantially assist or replace discretionary decision making.” The proposed rules clarify the meaning of each of these phrases.

According to the proposed rules, the term “machine learning, statistical modelling, data analytics, or artificial intelligence” means “a group of mathematical, computer-based techniques”:

1. that generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate’s fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; and

2. for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and other parameters for the models in order to improve the accuracy of the prediction or classification; and

3. for which the inputs and parameters are refined through cross-validation or by using training and testing data.

The proposed rules define “simplified output” to mean “a prediction or classification” that “may take the form of a score (e.g., rating a candidate’s estimated technical skills), tag or categorization (e.g., categorizing a candidate’s resume based on key words, assigning a skill or trait to a candidate), recommendation (e.g., whether a candidate should be given an interview), or ranking (e.g., arranging a list of candidates based on how well their cover letters match the job description).” The proposed rules specify that a “simplified output” does not include “the output from analytical tools that translate or transcribe existing text” or “transcribe a video or audio interview.”

Additionally, the proposed rules explain that “to substantially assist or replace discretionary decision making” means:

• “to rely solely on a simplified output (score, tag, classification, ranking, etc.), with no other factors considered”; or

• “to use a simplified output as one of a set of criteria where the output is weighted more than any other criterion in the set”; or

• “to use a simplified output to overrule or modify conclusions derived from other factors including human decision-making.”

Identification of the Types of Decisions Covered by the Law

According to the language of the law, the term “employment decision” means “to screen candidates for employment or employees for promotion within the city.” The proposed rules explain that the term “[c]andidate for employment” means “a person who has applied for a specific employment position by submitting the necessary information and/or items in the format required by the employer or employment agency.” The proposed rules also specify that “[s]creen” means “to make a determination about whether someone should be selected or advanced in the hiring or promotion process.”

Guidance Concerning the Elements of a Bias Audit

As potentially impacted employers and employment agencies have been aware, the new law specifies that use of AEDT to screen a candidate or employee for an employment decision is unlawful unless the tool “has been the subject of a bias audit conducted no more than one year prior to the use of such tool.” The proposed rules address crucial questions left open by the language of the law. Specifically, the proposed rules state “a bias audit required by § 20-871 of the Code must, at a minimum”:

(1) Calculate the selection rate for each category;

(2) Calculate the impact ratio for each category; and

(3) Where [a tool] classifies individuals into groups, the calculations in paragraphs (1) and (2) of this subdivision must be performed for each such classification.

The proposed rules further state that when the tool “scores applicants or candidates,” the bias audit “must, at a minimum”:

(1) Calculate the average score for individuals in each category;

(2) Calculate the impact ratio for each category[.]

Selection Rates, Impact Ratio, and Categories Evaluated in a Bias Audit

The proposed rules also provide guidance concerning the factors that are to be evaluated in a bias audit. Specifically, the proposed rules define the terms “selection rate,” “impact ratio,” and “category.”

“Selection Rate” is defined as

the rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification by an AEDT. Such rate may be calculated by dividing the number of individuals in the category moving forward or assigned a classification by the total number of individuals in the category who applied for a position or were considered for promotion.

“Impact ratio” is defined as either “(1) the selection rate for a category divided by the selection rate of the most selected category or (2) the average score of all individuals in a category divided by the average score of individuals in the highest scoring category.”

“Category” is defined in the proposed rules as “any component 1 category required to be reported by employers … as designated on the Equal Employment Opportunity Commission Employer Information Report EEO-1.”

Characteristics of an Independent Auditor

The law specifies that the term “bias audit” means “an impartial evaluation by an independent auditor.” The proposed rules explain that an “[i]ndependent auditor” must be “a person or group that is not involved in using or developing an AEDT.”

Requirements for Publicizing the Results of a Bias Audit

The law also specifies that once a bias audit is completed, “[a] summary of the results of the most recent bias audit of such tool” and the “distribution date of the tool to which such audit applies” must be “made publicly available on the website of the employer or employment agency prior to the use of such tool.”

The proposed rules clarify these requirements. Specifically, the proposed rules provide that the “careers or jobs section” of the website must contain the following information “in a clear and conspicuous manner”:

(1) the date of the most recent bias audit of such AEDT and a summary of the results, which shall include the selection rates and impact ratios for all categories; and,

(2) the distribution date of the AEDT to which such bias audit applies.

The proposed rules state that an employer or employment agency may satisfy the publication requirement through the placement of “an active hyperlink to a website containing the required summary of results and distribution date, provided that such link is clearly identified as a link to results of the bias audit.” Further, the proposed rules specify that the required information must remain posted “for at least 6 months after last using such AEDT for an employment decision.”

Under the law, “information about the type of data collected for the automated employment decision tool, the source of such data and the employer or employment agency’s data retention policy shall be available upon written request by a candidate or employee.” The proposed rules clarify that if such information is not available on the website of the employer or the employment agency, the website’s careers or job section must “in a clear and conspicuous manner” incorporate “instructions for how to make a written request for such information.” Furthermore, if a “candidate or employee who resides in the city” makes a request, an employer or employment agency must either provide:

• “written notice in person, via U.S. mail or e-mail within 30 days of receipt of a written request for such information”; or

• “an explanation to a candidate or employee for refusing to provide information” … [if] disclosure of information required by the notice would violate local, state, or federal law, or interfere with a law enforcement investigation.”

Required Notifications to Employees and Candidates for Employment

In addition to the generalized notice, the law requires notification to any employee or candidate who resides in New York City and who has “applied for a position for an employment decision.” The proposed rules include guidance for providing the required notices.

According to the proposed rules, notice to a candidate for employment who resides in the city may be provided by:

(1) Including notice on the careers or jobs section of its website in a clear and conspicuous manner at least 10 business days prior to use of an AEDT;

(2) Including notice in a job posting at least 10 business days prior to use of an AEDT; or,

(3) Providing notice to candidates for employment via U.S. mail or e-mail at least 10 business days prior to use of an AEDT.

The proposed rules specify that notice to an employee who resides in the city may be provided by:

(1) Including notice in a written policy or procedure that is provided to employees at least 10 business days prior to use of an AEDT;

(2) Including notice in a job posting at least 10 business days prior to use of an AEDT; or,

(3) Providing written notice in person, via U.S. mail or e-mail at least 10 business days prior to use of an AEDT.

The notices to candidates or employees “must include instructions for how to request an alternative selection process or accommodation.”

Next Steps for Employers

Employers may wish to evaluate any automated decision tools currently in use to determine whether those tools fall within the scope of the law, as clarified by the proposed rules, and, if they do, plan for how to comply with the law’s bias audit and disclosure requirements.

The New York City Department of Consumer and Worker Protection has scheduled a public hearing on the proposed rules for Monday, October 24, 2022, at 11:00 a.m. Interested parties may submit comments on or before the date of the hearing.