Inference of Median Subjective Threshold in Psychophysical Experiments ()
1. Introduction
In psychophysical experiments, it is always desirable to understand the mechanisms that give rise to the observed behavior [1]. Three types of psychophysical methods are commonly used in estimating the point of subjective equality and the just noticeable difference: the method of adjustment, the method of limits and the method of constant stimuli [2]. In the present study, we restrict our attention to the method of limits and the method of constant stimuli in psychophysical experiments.
We consider the response of a test subject upon a skin area being heated with a contact surface [3] [4] or with an electromagnetic wave [5]. In the case of contact heating, when the size and temperature of the contact surface are fixed (or in the case of electromagnetic heating, when the cross-section and power density of the beam are fixed), the strength of stimulus is solely described by the time period of heating, which we shall call the exposure duration. At any time, the test subject has two options: allowing the heating to continue or escaping from the heating when the heat induced exceeds the subject’s personal pain tolerance. The subject’s response by any given time is binary: escape or no escape. Once the subject moves away from the heating, that particular trial is completed and the time elapse from the start of heating to the escape is recorded as the escape time. The measured escape time in each trial provides a sample value of subjective threshold for a particular subject in a particular realization. The point of subjective equality (PSE) is given by the median of subjective threshold: when the exposure duration is set to the median subjective threshold, 50% of subjects are expected to escape. The objective of this study is to infer the median subjective threshold by adaptively adjusting test procedures and using the data collected in trials. We examine four different approaches and thereby identify an optimal approach for estimating the median subjective threshold.
2. Mathematical Formulation
Due to the biovariability and other uncertainties, the escape time of the test subject will be different from one trial to another. We model the subject response as solely determined by the stimulus (the exposure duration) and a subjective threshold that varies among test subjects and varies among different trial realizations for a given subject. The outside stimulus and the internal threshold are connected via the pain signal generated at the heat-sensitive nociceptors activated by the skin temperature increase [6]. The internal threshold on the nociceptors signal is manifested outside as the observable threshold on the exposure duration.
Let Z be the subjective threshold. We set
as the start of heating exposure. By time
, the cumulative stimulus (the exposure duration) is u and the subject response is determined by stimulus u and subjective threshold Z.
(1)
In a particular trial realization, Z is fixed but unknown. In (1), we write the subject response as a function of stimulus u to emphasize that in a trial, as the exposure duration u increases, the escape response will eventually occur. Over many subjects and over many trial realizations, mathematically Z is a random variable, reflecting the biovariability and other uncertainties. By definition, Z is the minimum exposure duration needed to induce the escape response for a particular subject in a particular trial. Since Z is naturally constrained by
, we use a Weibull distribution to model Z. The actual distribution of Z may be different. We will investigate the consequence of this discrepancy on the inference accuracy. The probability density function (PDF) of the Weibull distribution is
(2)
which is described by two parameters: shape parameter
and scale parameter
. The mean, median and standard deviation of U have the expressions
(3)
Here
denotes the median of Z, and
is the gamma function.
Two types of psychophysical experiments may be conducted using, respectively, the method of limits (ML) [7] [8] or the method of constant stimuli (MCS) [7] [9]. In the method of limits, the heating is kept on until the subject escapes. In other words, the stimulus (exposure duration) keeps increasing until escape. The recorded escape time gives the sample value of threshold for that particular trial realization. In this way, independent samples of the subjective threshold can be collected from multiple trials. In the method of constant stimuli, the stimulus (exposure duration) is prescribed. At the end of exposure, the escape may or may not occur, depending on the prescribed stimulus and the (unknown) value of threshold in that particular trial. The binary outcome, escape or no escape, is recorded in each trial.
Let z be the sample value of the subjective threshold in a particular trial. Under the condition that the test subject is unaware of the test type used in the trial, it is reasonable to expect that threshold z is independent of the test type. In a test using the method of constant stimuli, let u be the prescribed stimulus (exposure duration). For a random trial with stimulus u, under the assumption that Z has a Weibull distribution, the probability of escape is given by the Weibull psychometric function denoted by
, which is the cumulative distribution function (CDF) of the Weibull distribution.
(4)
The median subjective threshold
satisfies
. In the context of psychometric function
,
is the point of subjective equality (PSE), the stimulus level at which it is equally likely that the heat induced is above or below the subjective pain tolerance. If we view
as a dose-response relation,
is the half maximum response dose. We write the psychometric function
in terms of
and
.
(5)
The goal of this study is to estimate
from data/observations. We choose to infer
instead of
for several reasons:
·
is the point of subjective equality (PSE) in the psychometric function.
· If we measure samples of Z using the method of limits, the sample median is less susceptible to outliers, which may occur in real experiments.
· The median is preserved in any monotonic transformation, which allows us to work with random variable
to infer
in the adaptive Bayesian method. Notice that while Z is constrained by
, variable Y is unconstrained.
· In the adaptive Bayesian method, the convergence of the inferred median is independent of whether the assumed model is correct. In our simulation analysis, we will demonstrate that even when the assumed model deviates from the actual psychometric function, the inferred median still converges to the correct value. If we choose
as the inference variable, however, this convergence property is lost.
3. Inference Methods
We discuss four inference methods. Each method is based on different assumptions and thus is applicable in different situations.
3.1. Method 1: Sample Median
This method does not assume any particular distribution form of Z. It is applicable for estimating the median of any random variable. We measure independent samples of subjective threshold Z in psychophysical tests using the method of limits. The data set is
(6)
We use
to denote the inference result of method 1 for
.
(7)
3.2. Method 2: Maximum Likelihood Estimation with 2 Unknown Variables
Like method 1 above, the maximum likelihood estimation (MLE) requires data set
, consisting of measured independent samples of Z. MLE (2 variables) assumes that the subjective threshold Z has the Weibull distribution given in (2), with shape parameter
and scale parameter
both unknown. MLE (2 variables) is applicable only when Z has the Weibull distribution. MLE (2 variables), we infer
simultaneously by maximizing the likelihood and then calculate
from
. For mathematical convenience, let
. In terms of
, the likelihood function and the log-likelihood have the expressions:
(8)
We maximize the log-likelihood
to estimate
.
(9)
Once
and
are determined, the inference result of method 2 for
, denoted by
, is calculated using Equation (3) and
.
(10)
3.3. Method 3: Maximum Likelihood Estimation with 1 Unknown Variable
Like methods 1 and 2 above, method 3 requires data set
, consisting of measured independent samples of Z. This method assumes 1) the subjective threshold Z has the Weibull distribution of the form (2) and 2) the shape parameter
is known, which leaves the scale parameter
as the only unknown. MLE (1 variable) is applicable only when Z has the Weibull distribution and the true value of
is given. We write the log-likelihood in terms of
.
(11)
The log-likelihood has the derivative
The maximization problem of
has an analytical solution.
(12)
The inference result of method 3 for
, denoted by
, is calculated using (3)
(13)
3.4. Method 4: Adaptive Bayesian Method
Unlike methods 1-3 above, the adaptive Bayesian method does not require measured independent samples of Z. Rather it utilizes binary outcomes observed in tests using the method of constant stimuli (MCS). Suppose a sequence of n trials of the MCS type have already been completed with prescribed stimuli (exposure durations)
. The observed binary outcome
of each trial indicates either
(if no escape occurs) or
(if escape occurs).
(14)
Here
is the (unknown) value of subjective threshold in trial j. In trials of the MCS type, values of
are not fully revealed. Instead, only
or
is observed.
Following the approach used in the Quest package [10], we use variable
to quantify the stimulus. Thus, stimulus x corresponds to exposure duration
. Notice that
is mathematically unconstrained and covers a wide range of exposure duration u whenx is varied moderately. The data set from the n trials consists of n pairs of
where x is the stimulus and I is the corresponding outcome.
(15)
To infer
in a Bayesian framework, we treat
as a random variable and use a normal distribution as the prior.
(16)
For a trial with stimulus x (exposure duration
), under the assumption that the subjective threshold Z has a Weibull distribution, the probability of escape is described by the psychometric function
in (5)
The model psychometric function
has parameters
and
. In experiments, however, the probability of escape is governed by the actual psychometric function
, which may be different from
.
In the adaptive Bayesian method, we assume
is given and our goal is to infer
. For that purpose, we need to view the probability of escape as a function of
. We write
in terms of
and
as.
(17)
where
is parameter-free function defined as
(18)
To facilitate the inference of
, we shift
by its prior mean and consider
. The prior distribution of
is
(19)
(19) is our prior knowledge about
. When data set
from n trials becomes available, the posterior distribution of
is calculated according to the Bayes theorem.
(20)
where
is the Bernoulli distribution. A key component of the adaptive Bayesian method is to select the stimulus for the next trial based on the current posterior of
. Supposen trials have been conducted and the data set
is used to calculate the posterior
in (20). Let
denote the stimulus for the next trial. The optimal choice of
is the point of subjective equality (PSE) of the actual psychometric function where
. The PSE of the actual psychometric function is unknown. It is the inference variable. We use the best available estimate of PSE as the next stimulus. We set stimulus
to the posterior median of
, which reflects the prior of
and the observed outcomes of the preceding n trials.
(21)
Once stimulus
is prescribed, trial
is carried out and the observed binary outcome
is used to update the posterior distribution of
.
(22)
It is important to emphasize that the probability of escape in real experiments is governed by
, independent of the assumed model distribution.
We repeat the process of adaptively selecting stimulus x based on the current posterior distribution, carrying out the next trial and using the observed outcome to update the posterior distribution. The iterative process of (21)-(22) is repeated either for a specified number of trials or until the standard deviation of posterior
is less than a specified tolerance. When the adaptive Bayesian iteration is completed, the inference result of method 4 for
, denoted by
, is calculated as the posterior median of
(23)
4. Relative Errors of Inference Results: Direct Monte Carlo Simulations
We use Monte Carlo simulations to evaluate the inference accuracy for each of the four methods described in the previous section.
4.1. Measurement of Relative Error
We define the relative error as the root-mean-square (RMS) relative difference between the inference result and the true value. For each inference method, we carry out M rounds of Monte Carlo simulations. For methods 1 - 3, each round of simulations involves generating a data set of N samples and calculating an inference result from each simulated data set. For method 4, each round of simulations consists of repeating for N trials the Bayesian process of adaptively selecting the stimulus based on the posterior distribution, carrying out the trial and updating the posterior distribution according to the observed outcome. Let
·
be the true value of
and
·
be the inferred
, using method k in round i of simulations.
From the Monte Carlo simulations, we calculate the relative error of method k as
(24)
4.2. Parameters in Simulations
We use
rounds of Monte Carlo simulations in the calculation of each relative error (each entry in Table 1 and Table 2). We calculate the relative error of each inference method, for each prior and for each sample size of
, 20, 40 or 80 trials.
Before we select values of
, we point out that random variable Z, the exact and inferred (mean, median, std) of Z, are all proportional to scale parameter
. In particular, both
and
are proportional to
. Thus, the relative error is independent of
.
In simulations of this section, we use
and 5. For the convenience of setting up simulations, we set
to make
. Figure 1 displays the Weibull distributions and Weibull psychometric functions for three values of
. For a smaller value of
, the Weibull distribution is wider (the normalized standard deviation is larger).
4.3. Prior Distribution of
in Method 4
In the adaptive Bayesian method, an estimate of
is obtained via inferring
. The prior for
is a normal distribution:
. The corresponding prior of
is a log-normal distribution.
(25)
The prior (mean, median and standard deviation) of
are related to
Figure 1. Effect of shape parameter
. Left panel: 3 Weibull distributions with median = 1. Right panel: 3 Weibull psychometric functions with PSE = 1.
by
(26)
Suppose we know the prior median and standard deviation of
from existing data of measurements on the subjective threshold Z. Given
, we can build the normal prior of
by solving for
in (26)
(27)
For small
, the Taylor expansion of the RHS yields
(28)
The prior relative uncertainty of
is described by the ratio
. From the given
, the prior standard deviation of
is calculated using Equation (27). In Monte Carlo simulations, the prior mean of
is set to a random variable
where
is the exact median of
. This randomness of
in simulations is both realistic and necessary. In real applications, the exact value of
is unknown. In the adaptive Bayesian method, the inference result for
is given by the median of the posterior
. If we set
, then before any Bayesian updating, the prior distribution already gives the exact solution, which is unreasonable. Setting
is consistent with that the prior error of
is
.
4.4. Simulation Results of the Four Methods
In all simulations, data sets are generated using the actual distribution. In Section 4, the actual distribution and the assumed model distribution have the same Weibull form and the same shape parameter
. The median of the actual distribution,
, is set to 1 while the median of the model distribution,
, is the inference variable. Not all methods utilize the Weibull distribution form or the value of
.
· Method 1 (the sample median) does not assume any distribution form.
· Method 2 (MLE, 2 variables) assumes the Weibull distribution form of Z but leaves shape parameter
as an unknown variable.
· Method 3 (MLE, 1 variable) both assumes the Weibull distribution form and uses the value of shape parameter
.
· Method 4 (adaptive Bayesian) both assumes the Weibull distribution form and uses the value of shape parameter
.
The situation where the actual distribution deviates from the assumed model distribution will be investigated in Section 5.
The relative errors for
are reported in Table 1. Methods 1 - 3 are based on measured escape times in tests using the method of limits. Of these three methods, the sample median has the largest relative error (except for
). This is reasonable since the sample median does not assume any distribution form.
Accordingly, MLE (1 variable) has the smallest relative error since it assumes both the correct distribution form and the correct shape parameter. Method 4 is based on binary outcomes (escape or no escape) observed in tests using the method of constant stimuli. Intuitively, a binary outcome provides less information than a full sample of escape time. The relative error of method 4 is larger than that of method 3 even though they have the same assumptions. However, we will learn in Section 5 that method 4 is very robust with respect to an incorrect model while for method 3, an incorrect model is fatal. The relative error of method 4
Table 1. Relative errors of the four inference methods:
.
is comparable to that of method 1. This feature will be examined in more detail in Section 5. An interesting result of Table 1 is that with 20 trials (
), the relative error is well above 10% unless we use the correct distribution form and the correct shape parameter in method 3, and use a very narrow prior in method 4 (adaptive Bayesian).
The relative errors for
are reported in Table 2. As
increases, both the actual distribution and the model distribution (they are the same in Section 4) get narrower (see Figure 1). A narrower distribution reduces the relative errors in all methods.
4.5. Relative Error vs Sample Size N
We study numerically the relative error (
) of each inference method as the number of trials (N) is increased. The simulation results for
are shown in Figure 2. For each of the three methods based on measured samples of random variable Z (left panel), the relative error is proportional to
. Method 1 has the largest proportionality coefficient while method 3 has the smallest.
Method 4 (adaptive Bayesian) updates the prior distribution based on binary
Table 2. Relative errors of the four inference methods:
.
Figure 2. Relative error vs N for
. Left panel: three methods based on measured samples of Z in the method of limits. Right panel: adaptive Bayesian method based on observed binary outcomes in the method of constant stimuli.
outcomes observed in the method of constant stimuli. Unlike the three methods in the left panel, method 4 has a finite relative error (the prior) before any binary outcome is observed. Intuitively, the prior can be viewed as an existing set of
binary outcomes. A narrower prior corresponds to a larger existing set (larger
). Consequently, the relative error is expected to be proportional to
.
(29)
The right panel of Figure 2 confirms that the relative error of the adaptive Bayesian method is well explained by relation (29).
4.6. Effect of Shape Parameter β
We explore how the relative error changes as the shape parameter
of the Weibull psychometric function is varied. We reiterate that in all simulations of Section 4, both the actual and the model psychometric function have the same Weibull form and the same shape parameter
. The situation where the actual psychometric function deviates from the model will be studied in the next section.
Figure 3 compares the results of
and
. For
, the behaviors of relative error vs N are qualitatively the same as those for
. The proportionality coefficients are noticeably smaller for
, consistent with the observation that the Weibull distribution is narrower for
(see Figure 1).
5. Inference Errors When the Actual Psychometric Function Deviates from the Model
As demonstrated in Figure 3, relative errors of inference are smaller when
is larger. To reduce the inference error, one may attempt to use a large
in the model disregarding the true
of the actual psychometric function (or in the absence of knowing the true
). In this section, we focus on the two methods that require the value of
as an input: the MLE (1 variable) and the adaptive
Figure 3. Comparison of relative error vs N for
and for
. Left panel: MLE (2 variables) and MLE (1 variable). Right panel: adaptive Bayesian method.
Bayesian method. We examine the behaviors of relative errors in several situations where the model deviates from the actual psychometric function.
5.1. Situation 1: The Actual Psychometric Function and the Model are Both Weibull
When both the actual psychometric function and the model have the Weibull form, we write them as two functions of
with different shape parameters and medians.
where function
is defined in (18). In the actual psychometric function,
is the true shape parameter and
the true median of
. In the model psychometric function,
is the prescribed shape parameter and
the inference variable.
The results for the 4 cases of
are displayed in Figure 4. It is clear that when
, the MLE (1 variable) does not converge to the true median (left panel of Figure 4). Therefore, we arrive at the conclusion below regarding the method of MLE (1 variable):
· the
value cannot be set arbitrarily in the MLE (1 variable); and
· when the true
of the actual psychometric function is unknown, the method of MLE (1 variable) is simply not applicable.
One striking feature of the adaptive Bayesian method is that it converges to the true median even when a wrong
value is prescribed in the model (right panel of Figure 4). At any given N, the relative error is mainly influenced by the true
of the actual psychometric function. A smaller
corresponds to a wider actual distribution and leads to a larger relative error at any given N. We discuss separately
and
.
When the prescribed
in
is larger than the true
in
Figure 4. Relative error vs N when
. Left panel: MLE (1 variable). Right panel: adaptive Bayesian method. Both methods require an input of
.
, the model distribution is narrower than the actual distribution. When
deviates from
in the direction of
, the relative error becomes noticeably larger than that of
. In addition, the decay of relative error vs N no longer follows the trend of
. Fitting
to the relative error vs N for
yields an exponent of
instead of 0.5. The fitting curves respectively for
and for
have the expressions
When the prescribed
in
is smaller than the true
in
, the model distribution is wider than the actual distribution. When
deviates from
in the direction of
, the relative error increases only slightly from that of
. The relative error vs N is still approximately described by the law of
, with the coefficient slightly above that of
. The fitting curves respectively for
and for
are
In the right panel of Figure 4, the relative errors for
(triangles) are noticeably higher than those for
(diamonds), which are already relatively large due to the small
. In comparison, the relative errors for
(squares) are only slightly higher than those for
(circles), which are relatively small due to the large
. This observation suggests that when facing an uncertainty of
, the best strategy for minimizing the inference error is to use the lower end of the range,
, in the model. In this way, if
is near the lower end, then it is well captured by
; if
is near the higher end, then the relative error for
is small and the relative error for
is only slightly higher. Notice that the lower end of
corresponds to the wider end of the distribution width. In summary, when facing an uncertainty in the actual distribution width, the best strategy for minimizing the inference error is to use a wide distribution in the inference model.
5.2. Situation 2: The Actual Psychometric Function Is Gamma and the Model Is Weibull
We consider the situation where the model distribution of subjective threshold Z is Weibull while the actual distribution of Z is gamma, which has the form
(30)
where
is the shape parameter and
the scale parameter of the gamma distribution. Two Weibull distributions for
and three gamma distributions for
are compared in the left panel of Figure 5. In each distribution, the scale parameter is selected to make median = 1 so all distributions are aligned at the same median. In terms of
, the actual and the model psychometric functions are
where
is the lower incomplete gamma function defined as
As in the case of Weibull distribution, when the actual distribution of Z is gamma, the relative error of inferred
is independent of the scale parameter
.
We test the adaptive Bayesian method using the Weibull model with
on three actual psychometric functions of gamma distributions with respectively
, 8, and 16. Relative errors vs N are plotted in the right panel of Figure 5. The results clearly demonstrate that the adaptive Bayesian method converges to the true median even when the assumed model and the actual psychometric function are of different types.
Now we look at the most important reason why we choose to infer
instead of
. In the assumed model (Weibull), the mean is related to the median by
(31)
which varies with
. In the actual psychometric function, the relation between
Figure 5. Left panel: two Weibull and three gamma distributions with media = 1. Right panel: inference errors of the adaptive Bayesian method using Weibull model with
on three actual psychometric functions of gamma distribution with
, 8, and 16.
the mean and the median is in general different from (31). Given the convergence of inferred
, it follows that if we choose to infer
, then the inferred
does not converge to the true
. That is, if we choose to infer
, the convergence of inference is lost.
In the left panel of Figure 5, the three gammas span a range of distribution width. In the right panel of Figure 5, we carried out inferences using the Weibull model of
, which is more consistent with the wider end (
) of the three gammas. Next we try inferences using the Weibull model of
, which is more consistent with the narrower end (
) of the three gammas. Figure 6 compares the relative errors of the two strategies of selecting
in the inference model: 1) selecting a smaller
to accommodate the wider end in a possible range of the actual distribution width or 2) selecting a larger
to accommodate the narrower end.
The results indicate that it is more important to accommodate the wider end in a possible range of the actual distribution width. If the actual distribution has a large width, failing to accommodate it will lead to a significant increase in relative error (compare triangles and diamonds in Figure 6). On the other hand, if the actual distribution has a small width, failing to accommodate it will increase the relative error only slightly (compare squares and circles in Figure 6). In addition, the relative errors for the case of small actual distribution width are significantly below those for the case of large actual distribution width. All these observations suggest that we should accommodate the wider end in a possible range of the actual distribution width.
5.3. Relation between the Actual Relative Error and the Prediction from the Posterior Standard Deviation
In the adaptive Bayesian method, the posterior distribution of
not only produces a posterior median as the inference result for
, but also
Figure 6. Comparison of the two strategies of selecting
in the inference model: 1) smaller
(wider distribution) or 2) larger
(narrower distribution).
yields a posterior standard deviation
. When the sample size N is not very small, the posterior of
is approximately a normal distribution. A normal posterior for
translates to a log-normal posterior for
. We view the posterior standard deviation of
divided by the posterior median of
as the predicted relative error in the inferred
. This prediction is practically operational since it comes out naturally from the posterior. The predicted relative error is expressed in terms of
using (26).
(32)
Figure 7 compares the actual and the predicted relative errors when the adaptive Bayesian method using the Weibull model is applied on actual psychometric functions of the gamma-type (Situation 2). As we discussed in Figure 4 (Situation 1) and Figure 5 & Figure 6 (Situation 2), the actual relative error is mainly influenced by the width of the actual distribution of Z: the larger the width, the larger the error. The predicted relative error, on the other hand, is determined by the width of the model distribution.
The blue line and red line in the left panel of Figure 7 are almost identical to those in the right panel while the actual distributions (specified by
) are very different for these two panels. For an actual distribution that is relatively wide (
in the left panel), if we use
, the narrow model distribution significantly increases the actual error and significantly decreases the predicted error at the same time. Thus, using a narrow model distribution gives a false sense of inference accuracy: the predicted error is small while the actual error is much larger. We want to avoid this situation, which is another reason for not using a narrow model distribution. For an actual distribution that is relatively narrow (
in the right panel), if we use
, the wide model distribution increases the actual error only slightly and increases the predicted error significantly.
Figure 7. Comparison of the actual and the predicted relative errors. The model distribution is Weibull with
. The actual distribution of Z: is gamma with
.
Thus, using a wide model distribution yields a predicted error that is too conservative. A wide model distribution is less a problem for minimizing the actual error than for precisely predicting the error. Synthesizing the results of subsections 5.2 and 5.3, we draw two conclusions regarding the adaptive Bayesian method:
· when presented with the choice of two likely candidate model distributions, to minimize the actual error of inference in this uncertain situation, the best strategy is to select the wider one; and
· the error predicted from the posterior standard deviation is dominated by the assumed model distribution and is unreliable; the actual error of inference is mainly influenced by the actual distribution of the subjective threshold.
5.4. Performance Comparison of the Adaptive Bayesian Method and the Sample Median
The sample median is calculated from independent samples of the subjective threshold Z measured in the method of limits. The adaptive Bayesian method utilizes binary outcomes observed in the method of constant stimuli. Intuitively each binary outcome contains less information than a full sample value of Z: it only indicates whether the value of Z in that trial is below or above the prescribed stimulus. Given the difference in information content between these two types of data, we like to find out if the adaptive Bayesian method based on binary outcomes is less efficient in inferring
than the sample median calculated from full samples of Z.
Figure 8 compares the relative errors vs N of the two methods. In the Monte Carlo simulations, the actual distribution of Z is gamma with
. In the adaptive Bayesian method, the model distribution is Weibull with
. A fairly uninformative prior (
) and a moderately informative prior (
) are tested. Despite the fact that the assumed model is different
Figure 8. Comparison of the sample median and the adaptive Bayesian method. The actual distribution of Z is gamma with
. The assumed model is Weibull with
.
from the actual psychometric function, the Bayesian method is as efficient as the sample median in estimating
. For moderately large N, the relative errors of the Bayesian method are indistinguishable from those of the sample median, and are fairly independent of the prior. Thus, we conclude that for estimating the point of subjective equality (PSE) in psychophysical experiments, the method of constant stimuli is as efficient as the method of limits.
6. Concluding Remarks
We considered the psychophysical experiments in which a skin area of a test subject is exposed to an electromagnetic heating or a contact heating. Once the heat induced exceeds the subject’s personal pain tolerance, the subject escapes from the heating. In this stimulus-response problem, when the specifications of heating (the spot size and power density of the electromagnetic beam) are fixed, the stimulus is solely described by the exposure duration, and the escape response is governed by a subjective threshold that varies among different subjects and varies among different realizations for the same subject. Mathematically, the subjective threshold is a random variable (Z). In the context of psychometric function, the median subjective threshold is the point of subjective equality (PSE). We studied the problem of inferring the median subjective threshold in psychophysical experiments. There are two types of experiments: a) the method of limits, and b) the method of constant stimuli. In the method of limits, the heating is kept on until escape (i.e., the stimulus keeps increasing until escape). The recorded escape time gives a sample value of the subjective threshold for that particular trial. In this way, independent samples of the subjective threshold are measured. In the method of constant stimuli, the heating is kept on only for a prescribed duration (i.e., the stimulus is prescribed). The observed outcome is binary: escape or no escape. In the inference, we modeled the subjective threshold as a Weibull distribution which is described by shape parameter
and scale parameter
. The actual distribution of the subjective threshold may deviate from the assumed model.
We examined four methods for inferring the median subjective threshold: 1) sample median, 2) maximum likelihood estimation (MLE) with 2 unknown variables (
and
are both unknown), 3) MLE with 1 unknown variable (
is given and
is unknown), and 4) adaptive Bayesian method. Methods 1 - 3 are based on independent samples of subjective threshold measured in the method of limits. Out of methods 1 - 3, method 1 does not assume any distribution form; not surprisingly it has the largest inference error. Method 3 assumes both the Weibull distribution form and the exact value of
; it has the smallest inference error. However, when the assumed
value is wrong or the actual distribution deviates from the assumed Weibull model, method 3 produces an incorrect result.
A large part of our study was focused on the adaptive Bayesian method, which assumes a Weibull model distribution and uses it to extract information from binary outcomes observed in the method of constant stimuli. Based on the Monte Carlo simulation results, we draw several key conclusions about the adaptive Bayesian method.
· Even when the actual psychometric function deviates from the assumed model, the inferred
in the adaptive Bayesian method converges to the true value.
· If we choose to infer
(the mean subjective threshold) in the adaptive Bayesian method, this nice convergence property is lost.
· At any fixed N (number of trials), the actual relative error of the adaptive Bayesian method is mainly influenced by the width of the actual distribution of Z: a wider actual distribution leads to a larger relative error.
· When the width of the assumed model distribution is wider than that of the actual distribution, the actual relative error is fairly insensitive to the inconsistency. In contrast, when the width of the assumed model distribution is narrower than that of the actual distribution, the actual relative error increases significantly with the inconsistency.
· The predicted relative error from the posterior standard deviation is mainly influenced by the width of the assumed model distribution. Using a model distribution that is too narrow produces both a large actual error and a false sense of accuracy (small predicted error). On the other hand, using a model distribution that is too wide yields a predicted error that is too conservative.
· When facing an uncertainty in the actual distribution width, the best strategy is to select a smaller
in the Weibull model to accommodate the wider end in a possible range of the actual distribution width.
· The actual errors of the adaptive Bayesian method are indistinguishable from those of the sample median, and are fairly independent of the prior used.
In summary, the adaptive Bayesian method is as efficient as the sample median for estimating the point of subjective equality in psychophysical experiments. This efficiency of the Bayesian method is remarkable since the sample median requires measuring full sample values of the subject threshold in the method of limits. In contrast, the adaptive Bayesian utilizes the binary outcomes observed in the method of constant stimuli. In addition, the Bayesian method achieves this efficiency even when the actual psychometric function deviates from the assumed model. In the adaptive Bayesian method, the error predicted from the posterior standard deviation is unreliable; it does not reflect the actual error. The actual error is mainly determined by the actual distribution of Z while the predicted error is dominated by the assumed model. To minimize the actual inference error when the actual distribution width is uncertain, it is best to use a wider model distribution.
Acknowledgements and Disclaimer
The authors acknowledge the Joint Intermediate Force Capabilities Office of U.S. Department of Defense and the Naval Postgraduate School for supporting this work. The views expressed in this document are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government.