Social media platforms, and the broader internet in general, have become rich sources of data over the last several years; among the most intriguing of these data is online consumer reviews of products and services. Sometimes referred to as e-Word of Mouth, or eWOM, these reviews typically appear on retailers’ websites, review-specific sites, and social media platforms, though they can be found in many other places.

Online reviews can be used in litigation matters and, if employed appropriately, can be a useful source of data for expert witnesses. For example, in consumer fraud or product liability cases, online reviews can be analyzed to determine whether consumers identified and commented on alleged defects. In intellectual property matters, it can be useful to examine how frequently a patented product feature was mentioned, or whether the feature is considered valuable. In cases about consumer finance products, online comments can potentially gauge the extent of confusion among consumers related to those products. Some false advertising cases have been brought by companies that allege a competitor posted fake reviews that drove sales down. In many of these cases, the relevant question frequently is: “Did online reviews impact sales of the product or service at issue?”^[i]

Moving beyond a simple assessment of impact on sales

Academicians have been studying how online reviews can impact sales for years, producing papers such as Rosario, et al. (2016)^[ii]. However, a cursory review of academic papers to date makes it clear that assuming good reviews lead to higher sales while bad reviews cause declining sales is much too simplistic. A more nuanced assessment would be to analyze which specific features of reviews, if any, are likely to have an impact on sales volumes. For example, Rosario, et al. state that: “Some studies have shown that negative eWOM is detrimental and even more powerful in decreasing sales than positive eWOM is in increasing it; conversely, other studies have shown that the presence of negative eWOM increases product evaluations and sales.” Thus, the conclusions that can be drawn from online reviews is very much fact-specific.

Today, the use of online reviews in litigation is moving beyond the simplistic links between reviews and sales and into more complex issues and analyses. We look at some of them here.

Consistency: A threshold issue is whether basic assumptions about the link between online reviews and sales can be applied consistently in all transactions or purchases. This may not be the case. For example, online reviews may have little to no impact on impulse buys. In addition, consumers may attach different values to online reviews based on their own perspectives. Those who make purchases based on brand loyalty, for instance, may place little to no importance on reviews; consumers with little brand loyalty may find online reviews extremely valuable.

Elements of Reviews: An additional issue relates to how specific elements within online reviews may impact sales. For example, reviews frequently contain a rating system in the form of stars or other types of numeric values. In the academic literature, numeric and other ratings are sometimes referred to as the valence of reviews. But other elements beyond the valence can also be analyzed. In some instances the number or volume of reviews can have an impact on sales. In some instances, it can be important to consider whether reviews by verified purchasers or trusted influencers have spurred sales more than reviews by unknown consumers. Whether reviews come from professional or peer reviewers can also be critical. Finally, the actual text of reviews can boost or hurt sales, depending on case-specific circumstances.

Assessing Valence: The impact of valence can also be analyzed, though the appropriateness of any particular analysis will depend on the specific case issues. For instance, one can simply look at the ratio of one-star versus five-star reviews to see if that ratio has meaningfully changed over time. Alternatively, one could examine whether the average valence of reviews has significantly changed after an event that is important to the litigation. Several critical caveats should be considered when using such an approach, however. First, many sites may skew results and display their own average ratings that may bear no relation to the simple numeric averages in the reviews. It may be difficult or impossible to know what average rating a consumer may have seen previously. Another caveat is that early reviews of new products or services may be glowing because purchasers are hardcore fans of the company. These early ratings may or may not paint a meaningful picture of the quality of the product or service at issue.

Reviewing Content: The contents, or text, of reviews can provide additional support for legal opinions but analyzing text can be more time consuming than analyzing ratings. There are many options for analyzing text. One approach is to use human coders to assess each review. Coders can evaluate how frequently a purportedly important feature appears in reviews, or they can identify how frequently product attributes are mentioned. Potentially, these results can be used to demonstrate the relative importance or insignificance of a feature at issue in a litigation. Another approach is to prepare a coding scheme so coders can identify certain pre-selected themes that may be important to the case. These types of analyses are generally referred to as content analysis.^[iii]

Computer coding: In some cases, though, the sheer number of online sources and the huge volume of individual reviews can make human coding challenging or impossible. When this happens, computer programs can be used instead of (or alongside) human coders and can identify how frequently important words or phrases appear in reviews. More sophisticated programs can be used to assess the relative positivity or negativity (frequently referred to as the sentiment) of the reviews, a technique that is widely accepted in the academic literature. The results of any sentiment analysis should be used carefully, however, as human coders can typically identify sarcasm or other nuances in texts that elude even sophisticated automated techniques.

Sentiment Changes: As we stated earlier, it’s frequently important to assess how the valence or sentiment of reviews change over time or in relation to a specific litigation date. In such cases, it may be possible to use statistical techniques to determine whether a change in valence or sentiment caused sales to drop. Such analyses typically require the use of sophisticated econometric or other techniques to ensure that the expert has appropriately controlled for other factors that may have impacted sales.

Propensity to Purchase: How online reviews impact consumers’ propensity to buy may also be relevant in some litigation. Surveys make it possible to test whether propensities meaningfully change if consumers are presented with a two-star versus a five-star review of a product, for example. Such a survey technique is sometimes referred to as an A/B design: one group of respondents would see a product page with a two-star review while second group would see the product page with a five-star review. One can then analyze the propensity of each group to buy the selected product.

The views expressed in this article are solely those of the author, who is responsible for the content, and do not necessarily represent the views of Cornerstone Research. This article was first published by Law360.