The Consumer Education Foundation, a California-based nonprofit consumer advocacy organization, has filed a petition with the FTC asking it to investigate the use of “secret surveillance scores” in the U.S. marketplace.

Such scores are claimed to be the product of analytics companies that “amass thousands or even tens of thousands of demographic and lifestyle data points about consumers, with the help of an estimated 121 data brokers and aggregators who are able to purchase … our personal data from consumer-facing companies across the global marketplace.”  To generate a score, raw data (factors) about consumers is fed “into an algorithm designed to trawl through reams of data to detect consumer behavior patterns.”  The analytics firms are claimed to “use algorithms to categorize, grade, or assign a numerical value to a consumer based on the consumer’s estimated predicted behavior.  That score then dictates how a company will treat a consumer.  Consumers deemed to be less valuable are treated poorly, while consumers with better ‘grades’ get preferential treatment.”  According to the petition, “because the nature of the personal information being fed into algorithms is concealed, it is impossible for a consumer to know whether the Secret Surveillance Scores are based on inaccurate, outdated, or unreliable information.”

The petition describes a “consumer value score” that it claims “assigns each person and household a monetary value: how much that consumer or household is worth based on the predicted profit that they will generate for the company.”  In addition to identifying consumers who may be viewed as troublesome because of behaviors such as frequent calls to customer service, the petition states that a “more nefarious purpose suggested by some is the customer value scores’ reliance on an assessment of a person’s age, where they live, their income level, behavioral information, the number of bedrooms in their house, their credit cards, or their marital status—characteristics that frequently serve as surrogates for gender, race, or other categories that constitute unlawful discrimination under federal law.”  The petition asserts that customer value scores “are used by retailers to make instantaneous, automated judgments about a consumer that may result in consumers paying different prices for the same product based on how much profit the algorithm decides a particular consumer will produce.”  Other scores described in the petition include fraud scores, tenant scores, and employment scores, which are allegedly used to deny merchandise returns, housing, and jobs respectively.  The petition does not raise the use of surveillance scores in credit underwriting decisions.

The petition claims that the use of secret surveillance scores is an unfair and deceptive practice under Section 5 of the FTC Act.  It calls on the FTC to investigate how the scores are generated and applied and by which companies, which consumers are being targeted by companies using the scores, and the impact of the scores on consumers and the U.S. marketplace.  It asks the FTC to enjoin the use of the scores if it finds “that companies using and developing secret surveillance scores are violating Section 5 of the [FTC] Act.”

The use of algorithms (often referred to as “artificial intelligence” or “AI”) is coming under increasing scrutiny.  Early last month, Democratic Senators Elizabeth Warren and Doug Jones sent a letter to the CFPB, Federal Reserve, OCC, and FDIC expressing concern that fintech and traditional lenders using algorithms in their underwriting processes may be engaging in unlawful discrimination.  Last week, the Senate Commerce Committee’s Subcommittee on Communications, Technology, Innovation, and the Internet, held a hearing to “how algorithmic decision-making and machine learning on internet platforms might be influencing the public.”  Also last week, the House Financial Services Committee’s Task Force on AI held a hearing titled, “Perspectives on Artificial Intelligence: Where We Are and the Next Frontier in Financial Services.”

In June, we released a podcast titled, “Using artificial intelligence for consumer finance: a look at the opportunities and challenges.”   In the podcast, we discussed the opportunities and challenges created by the use of AI models in consumer financial services, including the benefits of explainable AI and its implications for the consumer financial services industry, especially for applications where understanding the model’s reasons for returning a score or decision are necessary.  Click here to listen to the podcast