THE POWER OF SENSORY DISCRIMINATION METHODS

Difference testing methods are extensively used in a variety of applications from small sensory evaluation tests to large scale consumer tests. A central issue in the use of these tests is their statistical power, or the probability that if a specified difference exists it will be demonstrated as a significant difference in a difference test. A general equation for the power of any discrimination method is given. A general equation for the sample size required to meet Type I and Type II error specifications is also given. Sample size tables for the 2-alternative forced choice (2-AFC), 3-AFC, the duo-trio and the triangular methods are given. Tables of the psychometric functions for the 2-AFC, 3-AFC, triangular and duo-trio methods are also given.     

BETA-BINOMIAL TABLES FOR REPLICATED DIFFERENCE AND PREFERENCE TESTS

Ennis and Bi (1998) discussed the beta-binomial (BB) model for replicated difference and preference tests. Based on the BB model, tables of the minimum number of choice responses to achieve significance at α≦ 0.05 are provided for replicated 2-AFC, Duo-Trio, 3-AFC and Triangular tests. The theory underlying the tables, how to use the tables, as well as some examples to illustrate their use are given. The tables can be used to evaluate the results of replicated difference and preference tests using forced choice methods.     

THE BETA-BINOMIAL MODEL: ACCOUNTING FOR INTER-TRIAL VARIATION IN REPLICATED DIFFERENCE AND PREFERENCE TESTS

Binomial tests are commonly used in sensory difference and preference testing under the assumptions that choices are independent and choice probabilities do not vary from trial to trial. This paper addresses violations of the latter assumption (often referred to as overdispersion) and accounts for variation in inter-trial choice probabilities following the Beta distribution. Such variation could arise as a result of differences in test substrate from trial to trial, differences in sensory acuity among subjects or the existence of latent preference segments. In fact, it is likely that overdispersion occurs ubiquitously in product testing. Using the Binomial model for data in which there is inter-trial variation may lead to seriously misleading conclusions from a sensory difference or preference test. A simulation study in this paper based on product testing experience showed that when using a Binomial model for overdispersed Binomial data, Type I error may be 0.44 for a Binomial test specification corresponding to a level of 0.05. Underestimation of Type I error using the Binomial model may seriously undermine legal claims of product superiority in situations where overdispersion occurs. The Beta-Binomial (BB) model, an extension of the Binomial distribution, was developed to fit overdispersed Binomial data. Procedures for estimating and testing the parameters as well as testing for goodness of fit are discussed. Procedures for determining sample size and for calculating estimate precision and test power based on the BB model are given. Numerical examples and simulation results are also given in the paper. The BB model should improve the validity of sensory difference and preference testing.     

COMPARISON OF METHODS FOR ANALYZING REPLICATED PREFERENCE TESTS

Preference testing is commonly used in consumer sensory evaluation. Traditionally, it is done without replication, effectively leading to a single 0/1 (binary) measurement on each panelist. However, to understand the nature of the preference, replicated preference tests are a better approach, resulting in binomial counts of preferences on each panelist. Variability among panelists then leads to overdispersion of the counts when the binomial model is used and to an inflated Type I error rate for statistical tests of preference. Overdispersion can be adjusted by Pearson correction or by other models such as correlated binomial or beta‐binomial. Several methods are suggested or reviewed in this study for analyzing replicated preference tests and their Type I error rates and power are compared. Simulation studies show that all methods have reasonable Type I error rates and similar power. Among them, the binomial model with Pearson adjustment is probably the safest way to analyze replicated preference tests, while a normal model in which the binomial distribution is not assumed is the easiest.     

THE DESIGN OF REPLICATED DIFFERENCE TESTS

We show that adding replications in replicated difference tests results in larger power and smaller variance when the number of assessors is fixed. On the other hand, when the number of total assessments is fixed, the power usually decreases and the variability increases whenever replications are considered instead of different assessors. The appropriate numbers of assessor needed to gain the same power and variability, respectively, when replications are used will be given. It is shown that the number of assessors might indeed be reduced, but this has to be paid for by an increasing total number of assessments. We show that two key models, namely the mixture binomial and a corrected version of the Beta-binomial model, are quite similar with respect to the properties of interest. We provide tables from which, according to her/his requirements, the investigator might find an appropriate setting with respect to the number of assessors and replications.     

CONTRASTING THE EFFECTS OF INGESTION AND EXPECTORATION IN SENSORY DIFFERENCE TESTS1

Paired comparison and triangle tests were used to contrast the effects of ingestion and expectoration in two distinct systems: added salt in kidney beans and nonfat milk with added milkfat. Probit analysis was used to determine panel difference thresholds. Analysis of variance was used to evaluate significant differences between the methods of sampling and for effects of fatigue due to ingestion of the sample. No significant differences between the two methods of sampling were found for either thresholds or fatigue effects.     

DIFFICULTIES AND A WAY OUT: A BAYESIAN APPROACH FOR SENSORY DIFFERENCE AND PREFERENCE TESTS

There are some theoretical difficulties in the philosophy and methodology of sensory difference and preference tests. The difficulties lie in the assumption that the subjects have the same response ability, which is in conflict with psychological ideas about the processes of perception and decision. A Bayesian approach may overcome these difficulties by treating the parameter of proportion as a random variable. This paper presents a Bayesian approach for analysis of data from such sensory tests.     

A THURSTONIAN VARIANT OF THE BETA-BINOMIAL MODEL FOR REPLICATED DIFFERENCE TESTS

The beta-binomial model is combined with a Thurstonian psychometric function to obtain estimates of the parameters of a distribution applicable to replicated difference tests. A method of estimating the variance of d'obtained from these tests is provided. A formula for determining sample size, which is composed of the number of trials (or panelists) and the number of replications, to determine d'is also given.     

COMPARISON OF CORRELATED PROPORTIONS IN REPLICATED PRODUCT TESTS

Product testing with multiple matched samples is often encountered. If the response for each sample is binary, the test involves comparison of multiple proportions of dependent samples. In the paper, a test statistic due to Bennett (1967, 1968) is discussed and extended to the replicated testing situation. An adjusted Bennett's statistic is proposed based on the Dirichlet-multinomial distribution. The conventional and adjusted Bennett's statistics are illustrated by a numerical example in product testing.     

VARIABILITY IN HEDONICS: INDICATIONS OF WORLD-WIDE SENSORY AND COGNITIVE PREFERENCE SEGMENTATION

This paper approaches inter-individual differences in the hedonics of food products as an indication of the existence of different groups of people with different likes and dislikes. Rather than considering the differences as an unpleasant fact of nature, to be ignored, the paper shows how these differences point to segments of consumers in the population. These segments enable the product developer to create different types of food products, in order to satisfy the different tastes. The paper, presenting an algorithm for segmentation of individuals into different groups, results from three studies with foods, and the extension of the particular segmentation algorithm into concepts (with three additional studies). The paper presents both new data and results from previously published research. These data are presented in a case history format.     

SENSORY DIFFERENCE TESTS: THURSTONIAN AND SSA PREDICTIONS FOR VANILLA FLAVORED YOGURTS

The Thurstonian prediction that judges will get a higher proportion of correct 3-AFC tests than triangle tests was confirmed using a yogurt system. As predicted, d' values for the triangle and 3-AFC tests corresponded. Sequential Sensitivity Analysis predictions for the sensitivity of various tasting sequences were confirmed for the 3-AFC test but not the triangle test. Further predictions that the 2-AFC is more sensitive than the 3-AFC were confirmed. This increase in sensitivity more than offset the greater statistical power of the 3-AFC. Difficulties of specifying the nature of the difference for the 2-AFC and 3-AFC tests were overcome using a warm-up procedure.     

基于耳石形态的鱚属鱼类鉴别

利用9个耳石形状指标对我国鱚属鱼类少鳞鱚、多鳞鱚及斑鱚共78个个体的矢耳石形态进行了多元统计分析,同时对3个群体的74个少鳞鱚矢耳石形态进行了判别分析。单因子方差分析结果显示,少鳞鱚与多鳞鱚矢耳石的9个形状指标均存在显著差异(P<0.05);反映耳石长、短轴差异程度的第1主成分与反映耳石三维特征的第2主成分解释了85.943%的形态差异;9个指标在种间判别分析中的综合判别正确率为93.6%,种内判别分析中的综合判别正确率为93.2%。研究表明形状指标法可以作为中国鱚属鱼类鉴别的有效手段。     

THE β-STRATEGY: AN ALTERNATIVE AND POWERFUL COGNITIVE STRATEGY WHEN PERFORMING SENSORY DISCRIMINATION TESTS

For discrimination tests not requiring the nature of the difference to be specified in the instructions, such as the triangle, duo-trio and same-different protocols, it is usually assumed that the decision rule used by the subject in order to generate an answer involves the comparison of the relative sensory distances between the samples. However, there is evidence that an alternative cognitive strategy, the β-strategy, involving the absolute categorization of the sensations could be used by the subject when performing these protocols. This paper introduces the characteristics of this alternative strategy by discussing its statistical power (higher than the traditional strategies) and by describing means of investigating which strategy is used by the subject during an experiment as well as illustrating the experimental conditions that might facilitate the use of this particular decision rule.     

USE OF PURCHASE PREFERENCE OPTIONS TO INCREASE NO PREFERENCE FREQUENCIES IN PLACEBO PREFERENCE TESTS

Besides responses to the different stimuli being tested in a paired preference test, responses to identical placebo stimuli can be used as a screening tool to identify biased consumers. Those consumers who give preference responses to identical stimuli can be assumed to be biased. Accordingly, only the data from unbiased consumers need to be considered for the different stimuli. The problem with this procedure is that the sample size is reduced. The goal of the present research was to see whether using options associated with purchase intent, elicited a greater number of No Preference responses to identical placebo stimuli. It was found that they did. The increase was large when the preference options implied exclusivity. In conditions where the preference strength options were not so strong, the frequency of No Preference responses dropped accordingly.

PRACTICAL APPLICATIONS

A problem with paired preference testing is the tendency of consumers to give false preferences, which produces the seriously misleading overestimation of the proportion of consumers who have preferences for one or other of the products being assessed. The placebo condition is an important control for alleviating this problem. The statistical analysis can be improved by finding a protocol that maximizes the proportion of No Preference responses in the placebo condition. The key finding here is that using purchase intent questions rather than preference questions may possibly provide a way of achieving this aim.     

EFFECT OF QUESTION ORDER ON SENSORY PERCEPTION AND PREFERENCE IN CENTRAL LOCATION TRIALS

In the running of consumer studies aimed at obtaining information about products and services, it has become increasingly popular to include some sensory type questions about the characteristics of the test product as well as the standard question about overall preference. The implications of this for the resultant data are complex, and as Sudman and Schwarz (1989) point out question order effects have been implicated as one of the major causes of unreplicated or unexpected findings in survey research. This study aims to establish the effect of question order on overall preference scores and sensory scores in consumer studies. Results indicate that there is a significant effect of asking sensory questions on overall preference ratings and the position of the preference questions (relative to the sensory questions) also has implications for overall ratings of preference.