Introduction

The Comcast ruling in 2013 and U.S. case law since then raised the bar for plaintiffs to establish a causal link between their theories of liability and actual harm.¹  In particular, the ruling requires that plaintiffs show, on a common basis, that consumers suffered harm attributable to the conduct of the defendant, and that plaintiffs are capable of determining harm in a way that is consistent with the particular theory of liability offered.²

In product liability matters, plaintiffs typically claim that the defendant misrepresented the true characteristics and qualities of the products at issue (either through false claims on product labels/advertising communications, or through lack of proper disclosures/omissions).  Plaintiffs then claim that, because of such misrepresentations, they did not receive the benefit of their bargain and overpaid for the products they purchased, and/or the purchased products diminished in value following disclosures of the alleged misrepresentations.³  In these cases, plaintiffs need to show that individuals who purchased the at-issue products (1) saw the alleged misrepresentations (exposure), (2) relied on the alleged misrepresentations in making their purchase decisions (reliance), or in the case of allegedly omitted information, that the failure to disclose omitted information materially impacted their purchase decisions (materiality), and (3) as a result, buyers paid higher market prices or would have purchased a different product if they were provided with the relevant information at the time of product purchase (impact).

In the following sections, we describe and discuss strengths and weaknesses of empirical tools from the fields of marketing, consumer behaviour, and economics that are typically employed by plaintiffs and defendants in product liability litigation in addressing issues relating to exposure, reliance, materiality, and impact, including analytical tools usually employed to assess economic harm according to overpayment and diminution in value theories of harm.

Survey Methods

Survey methods have been heavily employed in product liability matters.⁴  They are typically conducted by experts who specialise in the fields of marketing, consumer behaviour, and survey methodologies.  Below we provide specific examples of the types of surveys that have been used in these matters.

Surveys Relating to Consumer Behaviour and Purchase Decisions

Research in marketing and consumer behaviour sets out a framework to assess the effects of a disclosure or omission of an alleged misrepresentation on consumers’ purchase decisions.  Specifically, this research indicates that consumers can vary in their purchase processes and reasons.⁵  For example, some consumers may perform significant research and consider multiple information sources prior to purchasing a given product, while others may not.  Furthermore, while some consumers might rely on information from the manufacturer or from the sales representatives, others might rely on information from third parties (e.g., Edmunds.com, CNET, Consumer Reports ratings, etc.), friends and family, or their own past experiences with a given brand.⁶  As a result, while some purchasers of the at-issue product may have been exposed to the manufacturer’s communications containing the alleged misrepresentations, others may not.

Similarly, research in consumer behaviour indicates that consumers have different preferences for different features of the same product.⁷  This variation (or heterogeneity) in consumer preferences implies that certain product features may be strongly valued by some consumers, but not by others (who may prefer other product features).  For example, while buyers of a specific car model may value speed and horsepower above all other features, other buyers may consider fuel economy most important.  As a result, disclosure of information about specific features of a product may affect each buyer differently, and for buyers who do not consider these features important, such disclosures may not change their purchase decisions.

Given this setting, in a product liability consumer class action, a marketing expert can review industry sources and product reviews (e.g., Edmunds.com, CNET, Consumer Reports), defendants’ internal marketing studies, and existing customer surveys (conducted by defendants in their normal course of business) to empirically assess the information sources and purchase factors considered by putative class members.

When these sources are not available or are inadequate, the marketing expert can also design a survey using a representative sample of buyers of the product at issue.  The purpose of the survey would be to (1) uncover the sources of information consumers relied on in their purchase decisions and relative importance of these alternative sources of information, and (2) uncover the factors that consumers considered in their purchase decisions and the relative importance of each of those factors.  The results of these surveys can be used to assess whether and to what extent manufacturer advertising influenced purchasers’ decisions to buy the product at issue, the reasons individuals purchased this product, and whether and to what extent the purchasers cared about the misrepresented or allegedly defective feature.

Surveys Relating to Contested Marketing Communications

Research in marketing and consumer behaviour further indicates that the interpretation of marketing communications may differ considerably across consumers.⁸  As a result, consumers may take away different messages from the defendant’s communications, such as advertisements and product labels.  Using a representative set of the defendant’s advertisements or other marketing communications, a marketing expert can design surveys to empirically assess consumers’ perceptions of the main messages conveyed by the contested advertising.

A marketing expert can also design a survey to assess whether the allegedly false advertising messages were material to consumers’ purchase decisions.  This typically involves a survey design that includes a treatment group and a control group.  Respondents in the control group are typically shown the original advertisement or label, while respondents in the test group are typically shown the same advertisement or label without the challenged message (or in cases where defendants allegedly omitted material information, a test group can be shown the same stimulus but with the message included that has allegedly been omitted in the contested advertisement or label).  After reviewing the stimuli, the respondents in both groups are asked to indicate purchase intent (e.g., indicate their likelihood of purchasing the product based on a scale).  If the survey design is robust, any difference in purchase intent measures across test and control groups would likely be due to test group respondents’ exposure to the challenged messages.  If purchase intent is not different across the two groups, the challenged messages or omitted information is unlikely to be material.

Conjoint

Recently plaintiffs’ marketing and economic experts in product liability matters have increasingly proposed conjoint analysis as a method to estimate damages based on theories of harm, such as the benefit of the bargain, overpayment damages, and/or damages from the diminution in the product’s value following the disclosure of an alleged misrepresentation.⁹

The use of conjoint analysis has varied across cases.In certain cases, the plaintiffs’ expert opinion was limited to a proposal for a conjoint analysis, which outlined the general contours of the conjoint survey instrument without a formal implementation.¹⁰  In other cases, the plaintiffs’ expert executed the survey, conducted the conjoint analysis, and presented empirical results.¹¹  In such instances, these experts attempted to measure the alleged “price premium” plaintiffs paid for the at-issue product because purchasers were not aware of the alleged defect or misrepresentation at the time of purchase. 

Definition of Conjoint

Conjoint analysis is a survey-based methodology used to analyse consumer preferences for products and product features.¹²  The main premise of this methodology is that products are comprised of a multitude of features called “attributes”.  For example, cars have a myriad of attributes such as brand, body style, engine power, transmission type, fuel economy, and so on.  The theoretical underpinning of conjoint analysis is that consumers’ utility stemming from the purchase of a product is the sum of the utilities (or “part-worths”) originating from each of the attributes that comprise such a product.  The goal of conjoint analysis is to measure consumers’ preferences for each product attribute.

A conjoint survey contains questions requiring respondents to choose among hypothetical product profiles which vary across the product attributes specified in the conjoint.¹³  By examining respondents’ choices, conjoint attempts to estimate respondents’ stated preferences for each product attribute relative to other attributes, and the rate at which respondents are willing to trade off these attributes with each other and with price.  This procedure, if done correctly, ultimately delivers estimates of respondents’ willingness to pay (or WTP) for each product attribute, expressed in monetary values.¹⁴

Challenges in the Use of Conjoint in Product Liability Cases

In the following sections, we identify the main challenges relating to the use of conjoint analysis in product liability matters.First, we explain that while conjoint analysis, if done correctly, can provide average willingness to pay estimates, it is an inappropriate method to estimate an alleged “price premium”. Then, we identify the typical flaws that may affect the implementation of conjoint analysis in estimating WTP.

Conjoint Does Not Model the Supply Side of the Market and, at Best, Can Generate Willingness to Pay, Not Market Price Estimates

As we explain above, conjoint analysis was developed to estimate consumers’ stated preferences for products and their attributes.There are two fundamental issues that need to be addressed in applying conjoint analysis to estimate harm in product liability matters.First, conjoint analysis reveals stated preferences of the at-issue product and not actual preferences based on actual purchase transactions of the at-issue product.Second, if done correctly, conjoint analysis can at best estimate willingness to pay and not market price.The distinction between these two concepts is crucial – willingness to pay is determined by analysing consumers’ demand for a product, while market price is determined based on the interaction between demand and supply in the marketplace.

The fact that conjoint can at best measure demand and willingness to pay, and not market prices, is well established in the literature:

• “Choice-based conjoint (CBC) surveys … have become widely used … to predict the demand for consumer products.”¹⁵
• “WTP measures only a shift in the demand curve and not what the change in equilibrium price will be as the feature is added or enhanced.”¹⁶
• “In general, the WTP measure will overstate the change in equilibrium price.”¹⁷

Consequently, conjoint analysis is an inherently inappropriate methodology to determine the market price of a product without an alleged defect or alleged misrepresentation.In conclusion, the results from a conjoint analysis cannot and should not be used to determine economic damages according to the benefit of plaintiffs’ bargain or overpayment damages in product liability class actions.

In recent years, courts have reached contrasting conclusions when considering the validity of the conjoint analysis put forward by the plaintiffs’ experts and their assumptions regarding supply-side factors.Among others, a number of recent court rulings have recognised that conjoint analysis does not account for the supply-side of the market and, therefore, cannot be reliably used to calculate damages.For example, in Saavedra v. Eli Lilly & Co., the court did not certify the putative class and concluded that:

“[The] model looks only to the demand side of the market equation.  By looking only to consumer demand while ignoring supply, Dr. Hay’s [the plaintiffs’ expert] method of computing damages converts the lost-expectation theory from an objective evaluation of relative fair market values to a seemingly subjective inquiry of what an average consumer wants.”¹⁸

In Morales et al. v. Kraft Foods Group, Inc., et al. the court de-certified the class and reached the following conclusion:

“It is uncontested here that the conjoint analysis conducted by Bodapati [the plaintiffs’ expert] did not measure the market value of the Product either with the ‘natural cheese’ label or without it … [T]he evidence provided by Plaintiffs about their potential willingness to pay a premium due to the use of the ‘natural cheese’ label is insufficient to establish a basis for calculating restitution.”¹⁹

In other product liability class actions, however, courts accepted conjoint analysis.  For example, in In re MyFord Touch Consumer Litigation, the court appears to have misunderstood accounting for supply-side factors with assuming a fixed quantity of supplied products.²⁰  As mentioned earlier, a conjoint cannot account for any supply-side factors such as costs of production or competitors’ reactions, nor does it account for any of the actions that manufacturers could undertake in response to disclosure of an alleged defect instead of lowering prices, such as offering free repairs and recalls, or extending warranties.

Aggregate WTP Estimates from Conjoint Can Mask Individual Responses of No Impact or Irrational Preferences

Conjoint studies performed in product liability litigation can typically provide estimates for each respondent’s preferences and willingness to pay for the product features in the survey.²¹  However, in many circumstances, the plaintiffs’ experts measure the loss by calculating an average or median willingness to pay measure across respondents.  Even if an aggregate willingness to pay estimate may indicate that willingness to pay declines due to disclosure of an alleged defect, this may not be the case for many or most individual respondents.  In particular, some consumers may not be affected by the challenged conduct because, for example, they do not attach any value to the allegedly defective feature of the product.  An analysis of individual-level willingness to pay estimates can therefore demonstrate lack of common impact if, for many or most respondents, the estimated decline in willingness to pay due to the challenged conduct is zero.

Furthermore, an analysis of individual-level willingness to pay can reveal that some respondents exhibit an increase, rather than a decrease, in willingness to pay due to the challenged conduct, (for example, all else equal, these respondents would be willing to pay a higher price for a defective product compared to a non-defective one).  Such irrational preferences would call into question the reliability of the data generated by the conjoint survey.

Selection of the Relevant Population

As explained by Dr. Shari Diamond in the Reference Guide on Survey Research, identifying the appropriate population is a key step for every survey.²²  In particular, it is important that the conjoint relies on a representative sample of the target population to whom conjoint results should be extrapolated.  Disregarding this basic principle creates a fundamental disconnect between the plaintiffs’ theory of liability and the findings presented by the expert.

Realism, Confusion, and Bias in Conjoint

For the conjoint analysis to provide reliable estimates of consumers’ preferences, the conjoint survey instrument should be able to reasonably replicate the consumers’ purchase decision-making process.In order to achieve this goal, certain fundamental conditions must be met.First, the product attributes included in the conjoint must contain important drivers of consumers’ purchase decisions in the real world.²³  In other words, “the menus of products and their descriptions [should be] designed to realistically mimic a market experience”.²⁴  Exclusion of salient product attributes and inclusion of attributes that consumers do not consider important in real markets creates a so-called “focusing bias”, whereby the relative importance of the lesser valued attributes are increased.  In such cases, the estimated valuations for these attributes are inflated.²⁵

Second, the choice questions included in the survey instrument must be clear and unambiguous.Failure to meet this basic requirement undermines the validity and reliability of the data generated by the survey.26 Third, no aspect of a survey instrument should be leading or suggestive in a way that would unduly influence respondents’ responses.²⁷  If respondents’ attention is drawn towards a particular attribute or attribute level (e.g., because the language used to describe an attribute makes it stand out from the others), the data generated by the conjoint survey may be biased.   

Content Analysis

Plaintiffs and defendants can also perform a “content analysis” of advertising materials and other communications disseminated by the defendant in order to assess the pervasiveness and uniformity of the alleged misrepresentations in these communications.  Content analysis is the systematic, objective, and quantitative analysis of the characteristics of various forms of communication (including advertising messages), and is a well-accepted methodology used by academics in various fields, including consumer research.²⁸

This analysis is typically performed by two or more human coders, who are “blind” to the purpose of the project.  Content analysis may also be performed by a computer-aided text analysis.  In either case, the content of advertising materials and other communications are categorised according to a set of objective rules (or a coding scheme) developed by the expert prior to the coding exercise.  Content analysis is suitable for expert testimony in litigation because it is replicable and has an error rate that can be measured.²⁹  The use of content analysis in litigation has expanded in recent years with frequent implementations in false advertising and product misrepresentation claims as well as in other areas such as defamation and securities fraud.³⁰  For example, the plaintiff in Beef Products, Inc. et al. v. American Broadcasting Companies Inc. et al., a prominent defamation case, relied on an expert testimony, which included several content analyses that contributed to the plaintiff’s causation evidence.³¹

Regression Methods

Regression methods can be used in product liability matters in measuring economic loss or damages.  It is important to note that these approaches can estimate, at best, only an average effect, and do not address the question of whether a particular individual was harmed, especially in situations where there is wide dispersion in the prices paid for the at-issue products (as is the case in the automobile industry and other industries, such as consumer packaged goods and consumer electronics).  Below we focus on two specific regression methods that are commonly used: “diff-in-diffs” regression models and synthetic control method; and hedonic regression models.

“Diff-in-Diffs” Regression and Synthetic Control Method

In order to assess diminished value and benefit of bargain theories, economics experts may rely on statistical analyses of market data.  For example, in cases where an alleged misrepresentation is revealed to the public, an expert may analyse whether the prices of at-issue products declined in response to such information revelation through a methodology called “diff-in-diffs” or difference in differences regression.³²  This methodology requires identifying a product (or a set of products) that is similar to the at-issue product but is not affected by the alleged misrepresentation, and whose price trajectory tracks that of the at-issue product before disclosure (i.e., a “control product”).³³  In order to isolate the average change in price that is due to the alleged misrepresentation from normal changes in prices that are due to unrelated reasons (e.g., macroeconomic factors, industry specific factors), one may measure the average change in price after a disclosure for both the at-issue product and the “control product”, and then measure the difference across these price changes (thus, difference in differences). 

A critical assumption for the diff-in-diffs methodology to yield meaningful results is that at-issue products and “control” products are similar in all ways, except for the alleged misrepresentation.  In other words, the diff-in-diffs methodology requires a set of benchmark or comparison products against which to compare the at-issue product, and the methodology assumes that, but for the disclosure, the price of the product at issue would have evolved the same way as the benchmark products.

In many circumstances, it may be hard or impossible to find suitable benchmark products or to fully account for all of the differences between the product at issue and the benchmark products.  In these cases, experts can resort to another method closely related to diff-in-diffs, called the “synthetic control” method.³⁴  The main difference between the synthetic control method and the diff-in-diffs method is that the benchmark product is not just one product but a basket of products.  In other words, this method attempts to find the mix of benchmark products that most closely approximates the product at issue (in terms of price and other characteristics) prior to the event or disclosure at issue.

Again, it’s important to note that these approaches, if done correctly, only estimate an average effect and cannot answer the question of whether a particular individual was harmed by the challenged conduct.

Hedonic Regression

Hedonic regression methodology is based on the premise that a product is made of a multitude of attributes or features, and each of these features contributes to customers’ overall utility for a product.Hedonic regression uses econometrics techniques to determine the price premium or discount determined by the attributes or features of the product at issue.³⁵  Hedonic regression relies on actual sales data and product features and exploits the variation in products’ market prices and actual features to estimate how the presence or absence of these features adds to or subtracts from the market price of a product. 

In the context of product liability matters, there are several challenges in using hedonic regression methods to estimate benefit of the bargain or overpayment damages, and/or damages from the alleged diminution in the product’s value.For example, one needs to carefully consider and decide which product features to include since many products have a large number of features.If one or more critical features are omitted, hedonic regression models have been found to generate estimates that are biased and unreliable.³⁶

The views expressed in this chapter are solely those of the authors, who are responsible for the content, and do not necessarily represent the views of Cornerstone Research. This article was first published by the International Comparative Legal Guides.

Endnotes

1. _{See, for example, Order Denying Motion for Class Certification, Davidson et al. v. Apple, Inc., United States District Court for the Northern District of California, No. 16-CV-04942-LHK, May 7, 2018, pp. 19–20; Order Denying Plaintiffs’ Amended Motion for Class Certification, In re NJOY, Inc. Consumer Class Action Litigation, United States District Court for the Central District of California, No. CV 14-428-JFW, February 2, 2016, pp. 6–8.}

2. _{At the class-certification stage, “any model supporting a plaintiff’s damages case must be consistent with its liability case . . . [and] courts must conduct a rigorous analysis to determine whether that is so”.  Comcast Corp. et al. v. Behrend et al., 133 S. Ct. 1426, 1433 (2013).}

3. _{See, for example, Fourth Amended Class Action Complaint, Davidson et al. v. Apple, Inc., United States District Court for the Northern District of California, No. 5:16-cv-4942-LHK, January 3, 2018, ¶ 165; Second Amended Class Action Complaint, Elizabeth Callaway et al. v. Mercedes-Benz, LLC, United States District Court for the Central District of California, No. 8:14-cv-02011 JVS, June 8, 2015, ¶¶ 17, 53–54; Amended Class Action Complaint, Flynn et al. v. FCA US LLC et al., United States District Court for the Southern District of Illinois, No. 3:15-cv-855, December 22, 2015, ¶¶ 72–74; Second Amended Consolidated Class Action Complaint, In Re Chrysler-Dodge-Jeep Ecodiesel^® Marketing, Sales Practices and Products Liability Litigation, United States District Court for the Northern District of California San Francisco Division, No. 3:17-md-02777-EMC, May 16, 2018, ¶¶ 255–257; Third Amended Class Action Complaint, In re MyFord Touch Consumer Litigation, United States District Court for the Northern District of California San Francisco Division, No. 13-cv-3072-EMC, October 13, 2015, ¶¶ 323, 359, 373; First Amended Class Action Complaint for Damages, Oula Zakaria v. Gerber Products Co., United States District Court for the Northern District of California, No. 2:15-cv-0200-JAK, February 27, 2015, ¶ 91.}

4. _{Diamond, S. S. (2011), “Reference Guide on Survey Research”, in Reference Manual on Scientific Evidence 3^rd Edition, Washington, DC: The National Academies Press, 359–424 at p. 366.}

5. _{Peter, J. P., and J. C. Olson (2010), “Affect and Cognition and Marketing Strategy”, in Consumer Behavior and Marketing Strategy, New York, NY: McGraw-Hill, 36–65.}

6. _{Peter, J. P., and J. C. Olson (2010), “Affect and Cognition and Marketing Strategy,” in Consumer Behavior and Marketing Strategy, New York: McGraw-Hill, 36–65.}

7. _{Winer, R. S., and R. Dhar (2011), “Analyzing Consumer Behavior,” in Marketing Management, Upper Saddle River, NJ: Pearson Education, Inc., 87–123.}

8. _{See, for example, Mitchell, A. and J. Olson (1981), “Are Product Attribute Beliefs the Only Mediator of Advertising Effects on Brand Attitude?” Journal of Marketing Research 18, no. 3, 318–332; Mackenzie, S. (1986), “The Role of Attention in Mediating the Effect of Advertising on Attribute Importance”, Journal of Consumer Research 13, no. 2, 174–195; Krishnamurthi, L. and S.P. Raj (1985), “The Effect of Advertising on Consumer Price Sensitivity”, Journal of Marketing Research 22, no. 2, 119–129; Ford, G., D. Smith, and J. Swasy (1990), “Consumer Skepticism of Advertising Claims: Testing Hypotheses from Economics of Information”, Journal of Consumer Research 16, no. 4, 433–441.}

9. _{See, for example, conjoint analysis has been proposed or implemented in recent product liability class actions in the auto industry (In re GM Ignition Switch MDL Litigation, In re FCA EcoDiesel Litigation, Callaway et al. v. Mercedes-Benz, In re MyFord Touch Consumer Litigation, and Flynn v. FCA US LLC), in the food industry (Oula Zakaria v. Gerber Products Co. and Morales v. Kraft Foods Group Inc.), and in the consumer electronics industry (Davidson v. Apple Inc.).}

10. _{See, for example, Order Granting in part and Denying in part Defendants’ Motions for Summary Judgement and Plaintiffs’ Motion to Certify Class, Flynn v. FCA US LLC, United States District Court for the Southern District of Illinois, No. 15-CV-0855-MJR-DGW, July 5, 2018.}

11. _{For example, in the case Morales v. Kraft Foods Group Inc., the plaintiff expert proposed a conjoint analysis during the class certification stage and implemented the conjoint analysis after class certification was granted.  (In Chambers) Order Re Plaintiff’s Motion for Class Certification (DKT. 47), Claudia Morales, et al. v. Kraft Foods Group., Inc., et al., United States District Court for the Central District for California, No. LACV1404387JAKPJWX, June 23, 2015; (In Chambers) Order Re Defendant’s Motion to Exclude Plaintiff’s Survey and Expert Testimony of Dr. Anand V. Bodapati (Redacted DKT. 284, Unsealed DKT. 297); Defendants’ Motion for Decertification (Redacted DKT. 285, Unsealed DKT. 294); Defendants’ Motion for Partial Summary Judgement (Redacted DKT. 286, Unsealed DKT. 295), Claudia Morales, et al. v. Kraft Foods Group., Inc., et al., United States District Court for the Central District of California, No.LACV1404387JAKPJWX,, June 9, 2017.  Ultimately, the Court decertified the class in this case.}

12. _{Rao, V. R., Applied Conjoint Analysis, New York, NY: Springer.}

13. _{This form of conjoint analysis is known as Choice-Based Conjoint (or CBC).}

14. _{Ben-Akiva, M., D. McFadden, and K. Train (2018), Working Paper, “Foundations of Stated Preference Elicitation – Consumer Behavior and Choice Based Conjoint Analysis”, 1–144 at pp. 92–93.}

15. _{Ben-Akiva, M., D. McFadden, and K. Train (2018), Working Paper, “Foundations of Stated Preference Elicitation – Consumer Behavior and Choice Based Conjoint Analysis”, 1–144 at p. 9.}

16. _{Allenby, G., J. Brazell, J. Howell, and P. Rossi (2013), “Using Conjoint Analysis to Determine the Market Value of Product Features”, Proceedings of the Sawtooth Software Conference, October 2013, p. 342}

17. _{Allenby, G., J. Brazell, J. Howell, and P. Rossi (2013), “Using Conjoint Analysis to Determine the Market Value of Product Features”, Proceedings of the Sawtooth Software Conference, October 2013, p. 342.}

18. _{Order Denying Plaintiffs’ Motions for Class Certification Pursuant to federal Rules of Civil Procedure 23(b)(3) or 23(c)(4) [73,74], Saavedra  v. Eli Lilly & Co., United States District Court for the Central District of California, No. 2:12-CV-9366-SVW, Dec. 18, 2014, p. 5.}

19. _{(In Chambers) Order Re Defendant’s Motion to Exclude Plaintiff’s Survey and Expert Testimony of Dr. Anand V. Bodapati (Redacted DKT. 284, Unsealed DKT. 297); Defendants’ Motion for Decertification (Redacted DKT. 285, Unsealed DKT. 294); Defendants’ Motion for Partial Summary Judgement (Redacted DKT. 286, Unsealed DKT. 295), Claudia Morales, et al. v. Kraft Foods Group., Inc., et al., United States District Court for the Central District of California, No. LACV1404387JAKPJWX, June 9, 2017, pp. 28-29.}

20. _{Order Granting in Part and Denying in Part Defendant’s Motion for Summary Judgement, In re MyFord Touch Consumer Litigation, United States District Court for the Northern District of California, Case No. 13-cv-03072-EMC, February 14, 2018.}

21. _{For example, “Choice-Based Conjoint” is a popular type of conjoint that can estimate individual-level preferences and willingness to pay. Rao, V. R., Applied Conjoint Analysis, New York, NY: Springer, pp. 127–183.}

22. _{“One of the first steps in designing a survey or in deciding whether an existing survey is relevant is to identify the target population (or universe).  The target population consists of all elements (i.e., individuals or other units) whose characteristics or perceptions the survey is intended to represent.” Diamond, S. S. (2011), “Reference Guide on Survey Research,” in Reference Manual on Scientific Evidence 3^rd Edition, Washington, DC: The National Academies Press, 359–424 at p. 376.}

23. _{“[S]election of attributes and levels is a very crucial step in the design of conjoint studies … The scientific aspects arise from an understanding of the consumer’s choice process, more specifically salient attributes involved in the choice of an alternative by a majority of target consumers.” Rao, V. R., Applied Conjoint Analysis, New York, NY: Springer, p. 43.}

24. _{Ben-Akiva, M., D. McFadden, and K. Train (2018), Working Paper, “Foundations of Stated Preference Elicitation – Consumer Behavior and Choice Based Conjoint Analysis”, 1–144 at p. 11.}

25. _{See, for example, Schkade, D. A. and D. Kahneman (1998), “Does Living in California Make People Happy? A Focusing Illusion in Judgments of Life Satisfaction”, Psychological Science, 9, 340–346; Kahneman, D., et al. (2006), “Would You Be Happier If You Were Richer? A Focusing Illusion”, Science, 312, 1908–1910.}

26. _{Dr. Shari Diamond’s Reference Guide on Survey Research states that: “When unclear questions are included in a survey, they may threaten the validity of the survey by systematically distorting responses if respondents are misled in a particular direction, or by inflating random error if respondents guess because they do not understand the question.  If the crucial question is sufficiently ambiguous or unclear, it may be the basis for rejecting the survey.”  Diamond, S. S. (2011), “Reference Guide on Survey Research”, in Reference Manual on Scientific Evidence 3^rd Edition, Washington, DC: The National Academies Press, 359–424 at p. 388.}

27. _{“[T]he wording of a question … can be leading or non-leading, and the degree of suggestiveness of each question must be considered in evaluating the objectivity of a survey.”  Diamond, S. S. (2011), “Reference Guide on Survey Research”, in Reference Manual on Scientific Evidence, 359–424 at p. 393.  “[I]n assessing the validity of a survey, the judge should take into account the following factors: whether the questions asked were … not leading.”  Federal Judicial Center (2004), Manual for Complex Litigation, Fourth Edition, S. Marcus et al. eds., Washington, DC: Federal Judicial Center, p. 103.}

28. _{Neuendorf, K. (2016), The Content Analysis Guidebook, Second Edition, Los Angeles, CA: Sage Publications, pp. 1–3; Gross, B. L., and J. N. Sheth (1989), “Time-Oriented Advertising: A Content Analysis of United States Magazine Advertising, 1890-1988,” Journal of Marketing 53, no. 4, 76–83.}

29. _{Tichy, L.K. and A. Shakotko, “Litigators, Wrangle That Data With Content Analysis”, Law360, (November 2017).}

30. _{Tichy, L.K. and A. Shakotko, “Litigators, Wrangle That Data With Content Analysis”, Law360, (November 2017).}

31. _{Tichy, L.K. and A. Shakotko, “Litigators, Wrangle That Data With Content Analysis”, Law360, (November 2017); Beef Products Inc. v. American Broadcasting Companies Inc., No. 12-cv-292, South Dakota Circuit Court, First Circuit, Union County.}

32. _{See, for example, Woolridge, J.M., Introductory Econometrics: A Modern Approach, Fourth Edition, Mason, OH; South-Western Cengage Learning, at pp. 450–454.} 

33. _{Abadie, A., A. Diamond and J. Hainmueller (2010), “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program”, Journal of the American Statistical Association 105, no. 490, 493–505 at p. 493.  See also, Card, D. and A. B. Krueger (1994), “Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania”, The American Economic Review 84, no. 4, 772–793.}

34. _{Abadie, A., A. Diamond and J. Hainmueller (2010), “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program”, Journal of the American Statistical Association 105, no. 490 493–505.}

35. _{See, for example, de Haan, J. and E. Diewert (2013), “Hedonic Regression Methods”, in OECD, et al., Handbook on Residential Property Price Indices, Eurostat, Luxembourg.}

36. _{Greene, W. H. (2012), Econometric Analysis, Seventh Edition, New Jersey: Pearson, pp. 96–97; de Haan, J. and E. Diewert (2013), “Hedonic Regression Methods”, in OECD, et al., Handbook on Residential Property Price Indices, Eurostat, Luxembourg, p. 51.}