As previously discussed here, the use of artificial intelligence in the workplace has drawn scrutiny from regulatory bodies and activist groups who have expressed concern that such technology may not neutrally screen applicants without regard to protected characteristics.

To address these concerns, the New York City Council introduced a bill (Int. 1894-2020) on February 27, 2020, which would regulate the sale of “automated employment decision tool[s]” that filter candidates “for hire or for any term, condition or privilege of employment in a way that establishes a preferred candidate or candidates.”

Specifically, the bill aims to require: (1) technology companies to conduct annual bias audits before selling such tools in NYC; and (2) companies who use such tools to notify each candidate within thirty (30) days of screening of the specific tool used to evaluate them, as well as the qualifications or characteristics considered by the algorithm.

If passed, the bill would take effect on January 1, 2022, and require proprietors of automated employment decision-making software to provide the purchaser the results of the annual bias audit and a notice indicating that the tool is subject to the provisions of the law.

Notably, the bill expressly reserves the NYC Commission on Human Rights’ ability to enforce the NYC Human Rights Law as it relates to the use of this software, which underscores the potential legal risks associated with doing so (including but not limited to memorializing the results in an annual bias audit).

The NYC bill is just one example of recent efforts undertaken by state and local governments and agencies to regulate AI due to bias concerns. For example, on January 1, 2020, the Illinois’ Artificial Intelligence Video Act took effect which regulates employers’ use of algorithms to analyze video interviews. More recently, the Equal Employment Opportunity Commission announced its investigation of two (2) cases involving alleged algorithmic bias in recruitment.

To mitigate against the potential risks presented by this rapidly-evolving legal landscape, employers who are considering whether (and how) to use machine-learning software in the workplace should heed caution and consult counsel before doing so.