Extracting Human Reaction Time from Observations in the Method of Constant Stimuli ()
1. Introduction
We consider a subclass of psychophysical experiments [1] [2] satisfying the properties: 1) the power of a stimulus source is fixed; the total amount of stimulus applied on the test subject is specified by the duration of exposure to the stimulus source; 2) the test subject’s outcome is binary: positive/null response; a positive response has an associated observable action while a null response has no action; 3) the subjective threshold on the exposure duration for producing a positive outcome is a random variable, fluctuating from one exposure test to another; and 4) if it occurs, a positive response is initiated at the time when the subjective threshold is reached; the time of observed actuation of positive response is delayed from the time of its initiation; this time delay is the human reaction time.
A possible situation for the type of psychophysical experiments described above is the occurrence of flight action when a test subject is exposed to a millimeter wave beam [3] [4] [5]. During the exposure, the electromagnetic energy is absorbed by the skin [6]. The absorbed energy goes into increasing the skin temperature. Thermal nociceptors in the skin are activated upon the local temperature reaching the activation temperature [7]. The electrical signal produced by nociceptors is proportional to the total number of nociceptors activated [8] [9]. When the pain signal received by the brain is strong enough, the brain issues instructions for the muscles to carry out a flight action (evading the beam and/or turning off the beam power) [3] [4]. For a millimeter wave beam of fixed specifications, both the number of nociceptors activated and the electrical signal produced by nociceptors increase with the exposure duration. As a result, there is a subjective threshold on the exposure duration for the pain signal to reach the threshold to initiate the flight action [5] [10]. This subjective threshold is a random variable, reflecting the underlying biovariability. It fluctuates from one test subject to another, from one exposure test to another on the same subject [11]. The observed actuation time of flight action, if it occurs, is delayed from the time when the pain signal reaches the threshold for initiating the flight action: it takes time for the pain signal to propagate from the exposed skin to the brain, for the instruction signal to propagate from the brain to muscles, and for muscles to actuate the flight action [12] [13]. This time delay is the human reaction time. These time quantities are illustrated in Figure 1.
In the method of limits (MoL) [14] [15], the beam is kept on until the observed occurrence of flight action. In each MoL test, flight action occurs eventually and the time of observed flight action is recorded in the data set. The time of observed flight action is the sum of 1) the random subjective threshold on the
Figure 1. Flight action is induced by an exposure duration exceeding the fluctuating subjective threshold. The time of observed flight action is delayed from its initiation at the subjective threshold by the human reaction time.
exposure duration for inducing flight action; and 2) the human reaction time, both of which are unknown. If we know one of these two quantities, we can calculate the other from the data. When both of these two quantities are unknown, the data set of observed flight action times from MoL tests does not allow us to extract the subjective threshold and the human reaction time simultaneously.
In the method of constant stimuli (MoCS), the exposure duration is prescribed before the start of beam power [14] [16]. The exposure may or may not lead to flight action, depending on whether the realized sample of subjective threshold in that test (a random sample) is lower or higher than the prescribed exposure duration. The binary outcome regarding the occurrence of flight action is recorded in the data set. In addition, if flight action occurs, the observed time of its actuation is recorded in the data set. We will see that this data set from MoCS tests allows us to extract both the median subjective threshold and the human reaction time. First, we notice that the data set of binary outcomes is independent of the human reaction time; an adaptive Bayesian method has been proposed to design the MoCS tests and to infer the median subjective threshold from the observed binary outcomes [17]. This has been done in our previous study of millimeter wave exposure tests [11]. In the current study, we focus on extracting the human reaction time based on the data of observed flight action times collected in a sequence of MoCS tests designed by the adaptive Bayesian method [17].
The rest of this paper is organized as follows. In Section 2, we introduce mathematical notations and models for describing the process of exposure tests and the associated observations. In Section 3, we lay out the inference formulations and methods. In Section 4, we carry out Monte Carlo simulations to generate artificial data sets to test the performance of the inference method proposed. In particular, we test the situation where the underlying true model for data generation is different from the inference model. In Section 5, we discuss the main results and conclusions obtained.
2. Models of Exposure Tests and Observations
2.1. Method of Constant Stimuli (MoCS)
We consider the situation where a test subject is exposed to a millimeter wave beam in experiments. During exposure, a fraction of the beam power is absorbed into the skin and it heats up the tissue underneath the skin surface. When the beam power is relatively high, the relevant thermal effect of beam exposure takes place within a short time period. Over that short time period, the effect of surface radiation cooling and surface convection cooling is negligible; the effect of blood flow cooling is also negligible. Here the relevant thermal effect refers to heating the skin tissue to activate the thermal nociceptors. Under these assumptions, when exposed to a beam of constant (time-invariant) high power, within the relevant time frame, the skin temperature increases monotonically with time [18] [19].
Thermal nociceptors are activated in the region where the local temperature is increased above the activation temperature, leading to a heat sensation. As the activated volume gets larger, the heat sensation becomes stronger. Eventually, when the heat sensation exceeds the test subject’s tolerance, the subject quickly moves away to avoid the beam and/or quickly turns off the beam power (we shall call this the flight action of the test subject).
In the method of limits (MoL), the beam power is kept on and steady until flight action is materialized/observed, upon which the beam power is turned off either by the experimenter or by the test subject. The actual duration of exposure (from the start to the end of beam power) varies with the test subject’s personal tolerance on heat sensation, which fluctuates from one test subject to another, from one exposure test to another on the same subject.
In the method of constant stimuli (MoCS), in each test, the exposure duration is predetermined before the start of beam power. After the exposure of prescribed duration, flight action may or may not occur. In each individual test, the binary outcome regarding the occurrence of flight action is recorded. When it does occur, the time of observed flight action is also recorded, which contains the human reaction time.
2.2. The Occurrence of Flight Action
We view a millimeter wave exposure as a stimulus response process. The binary response is the occurrence of flight action. In the general situation, the stimulus is described by several variables: the power density of the beam used, the beam spot size, and the duration of exposure. When the power density and beam spot size are fixed, the stimulus is solely described by the exposure duration. Corresponding to the personal tolerance on heat sensation, there is a subjective threshold on exposure duration for inducing flight action. Similar to the case with the personal tolerance on heat sensation, the subjective threshold on exposure duration is a random variable, fluctuating from one test subject to another (lateral biovariability), from one exposure test to another on the same test subject (longitudinal biovariability). In a test, if the prescribed exposure duration exceeds the realized sample value of subjective threshold in that particular test, flight action occurs. If it occurs, the time of observed flight action is delayed from the time of its initiation at the time of subjective threshold. At the time of subjective threshold, the exposure has activated enough thermal nociceptors and generated a sufficiently large electrical signal that will eventually result in flight action even if the beam power is turned off right at that time instance. The time delay from the initiation to the actuation of flight action is caused by that the pain signals travel from nociceptors via neuron fibers to brain; the brain generates instruction signals; the instruction signals travel from brain to muscles; and finally the muscles carry out the flight action. This time delay is referred to as the human reaction time.
To facilitate the discussion, we list mathematical notations for the quantities and random variables introduced above.
·
: prescribed exposure duration, a predetermined quantity in each test
·
: prescribed exposure duration in test j.
·
: subjective threshold on exposure duration for inducing flight action, a random variable. Here
represents the randomness due to biovariabilities.
·
: median of the subjective threshold for flight action.
·
: standard deviation of the subjective threshold for flight action.
·
: the realized sample value of random variable
in test j.
·
: human reaction time, a random variable independent of
.
·
: median of the human reaction time. In this study, we focus on the case of
= deterministic =
(i.e., the human reaction time is an unknown but deterministic quantity).
·
: binary outcome regarding the occurrence of flight action, a binary random variable.
·
: occurrence;
: no occurrence.
·
: the realized sample value of
in test j.
·
: time of observed flight action, a conditional random variable.
is defined only if
(i.e., if flight action occurs).
·
: the realized sample value of
in test j with
.
The time quantities described above are illustrated in Figure 1. In the framework of stimulus response, the binary outcome
is completely determined by the prescribed exposure duration
and the random subjective threshold
.
(1)
Mathematically, binary random variable
is derived from random variable
with the prescribed exposure duration
as a parameter. The stimulus response function describes the probability of flight action vs the prescribed exposure duration.
(2)
The probability of flight action,
, is an increasing function of
. When
is right at
, we have
: it is equally likely for
to induce or not induce flight action.
In Figure 2, we illustrate four representatives of possible realizations for an exposure test. They are not exhaustive. The binary outcome
is determined by the relation between
and
. Flight action occurs when the subjective threshold
is below the prescribed exposure duration
. Since
is a random variable,
is possible and uncertain for any given
in the neighborhood of median subjective threshold
. A lower value of
Figure 2. Four representative realizations of an exposure test. The binary outcome (I) is determined by the relation between the subjective threshold (
) and the prescribed exposure duration (
). When flight action occurs, its actuation is at time
.
prescribed
may induce flight action in some tests while a higher value of prescribed
may fail to induce flight action in some other tests. Of course, the probability of
increases with the prescribed
. In the case of
, flight action occurs and is irreversibly initiated at the time of subjective threshold
; the flight action is actuated/observed at time
; the time elapsed between the initiation and the actuation,
, is the human reaction time.
2.3. The Time of Actuation/Observation of Flight Action
If flight action occurs, the time of its actuation/observation,
, is the sum of
and
(a deterministic quantity in the current study). The time of observed flight action
is defined only when
.
(3)
Note that the probability of
depends on the prescribed exposure duration
. Consequently, a description of conditional statistics of
is incomplete unless the value of
is provided. For conciseness, in notations for the conditional statistics of
, we shall omit
since the condition
is always implied when discussing
. Instead we focus on specifying
. For example, we use the notation
(4)
The conditional density
is heavily influenced by the prescribed exposure duration
. Figure 3 compares
for two values of
. It is clear that the conditional density varies with
. In particular,
is
Figure 3. Schematic graphs of conditional density
for
(bottom) and for
(top). It is clear that
is an increasing function of
.
an increasing function of
, which demonstrates that a description of
is incomplete unless
is specified.
In test j, neither the subjective threshold for flight action
nor the human reaction time
is directly measurable. Instead, the observed binary outcome
along with the prescribed exposure duration
provides a binary description of
.
(5)
In the case of
(flight action occurs), the time of observed flight action
gives the sum of
and
, both of which are unknown in our study.
(6)
In this study, we aim at inferring the statistics of subjective threshold
and the human reaction time
, from data measured in the method of constant stimuli (MoCS). First, we infer
from the data set of prescribed exposure durations and the corresponding binary outcomes
, collected in a sequence of exposure tests. As described in our previous study [11], the exposure tests are designed sequentially and adaptively in a Bayesian framework, based on the test results already collected. The inference of
is based on the maximum likelihood estimation (MLE) formulation using a Weibull inference model. We will study the robustness of inferring median subjective threshold
when the underlying true model governing the data generation is different from the assumed Weibull inference model.
Once
is obtained, we infer
and
from the data set of observed flight action times
. The inference of
is also based on the maximum likelihood estimation (MLE) formulation using a Weibull inference model. The primary objective is to infer the human reaction time
whereas
is an unknown variable in the inference formulation. Again, we will study the robustness of inferring
when the underlying true model governing the data generation is different from the assumed Weibull inference model.
3. Inference Formulations and Methods
3.1. Data Measured in the Method of Constant Stimuli
In the method of constant stimuli, in testj, the exposure duration
is prescribed; after the exposure, the observed binary outcome
regarding the occurrence of flight action is recorded. If
(flight action occurs), the time of observed flight action
is stored in data; if
(flight action does not occur), data entry
is undefined (empty). The statistics of
is meaningful only when conditional on
. The data set, measured in the method of constant stimuli, has the form
(7)
where N is the number of exposure tests. Again, the data entry
is defined only in those tests with
. Below we infer, from data set (7), the statistics of subjective threshold
for flight action and infer the deterministic value of the human reaction time
. It is worthwhile to point out that neither
nor
is directly measured in the data. In test j, the observed binary outcome
provides a binary description of sample
in that particular test, as described in (5). In the case of
, the time of observed flight action
gives the sum of
and
in that particular test, as given in (6).
We infer
and
in two steps as described in Figure 4. The inference of
uses only the data set of binary outcomes
which is independent of the human reaction time
. The subsequent inference of
Figure 4. Two inference steps for estimating
and
.
utilizes the estimated
from step 1 and the data set of observed flight action times
.
3.2. Inference Model for tc
In our inference formulation, we use the Weibull distribution to model
, the random subjective threshold for flight action. The PDF and CDF of the Weibull distribution are
(8)
(9)
where k is the shape parameter and
the scale parameter in the Weibull distribution. It is important to point out that the underlying true distribution of
is unknown. We simply use the Weibull distribution as the inference model in our formulation. In Monte Carlo simulations, we will study the performance of this Weibull-based inference when the underlying true model of
deviates from the Weibull distribution.
The mean, median, and standard deviation of
are
(10)
(11)
where
is the gamma function defined as
(12)
The mapping from
to
is invertible.
(13)
Numerical computation shows that
defined in (13) is an increasing function of u and thus is invertible. The inverse mapping allows us to specify the Weibull distribution using the median and the standard deviation
. This parameterization is necessary when we test the inference performance on artificial data sets generated from 6 distributions (including the Weibull distribution) as the underlying true model of
. In numerical tests, data sets generated from all distributions are constrained by a given median and a given standard deviation.
The conditional density of
given
is
(14)
The conditional density of
given
is
(15)
The conditional density (15) provides a proper mathematical framework for connecting 1) the observed flight action time
in a test with flight action
, 2) the prescribed exposure duration
, and 3) the unknown deterministic human reaction time
. It serves as a key component for inferring
from data
.
3.3. Inference of mc
As illustrated in the top row of Figure 4, we use the method of maximum likelihood estimation (MLE) to infer
from a date set
where
is the prescribed exposure duration and
the binary outcome in test j. Under the Weibull inference model, given
, the probability of observing binary outcome
is given by
(16)
where
is the CDF of the Weibull distribution given in (9).
contains two parameters:
. In the inference of
, we fix the shape parameter at
and focus on estimating the scale parameter
. As we discussed in our previous study [11], fixing k at a wrong value, especially at a smaller value (corresponding to a wider Weibull distribution), will not affect the inference accuracy of the median
.
To formulate the MLE method, we include inference variable
explicitly in the notation of Weibull CDF:
. Given data set
of binary observations from N exposure tests, the log-likelihood as a function of
is
(17)
The maximum likelihood estimate for
is
(18)
Once
is obtained,
is calculated according to (10)
(19)
3.4. Inference of (mR, sc)
After
is estimated, we use the method of maximum likelihood estimation (MLE) to infer
, the deterministic human reaction time and the standard deviation of
. The inference is based on a date set
where
is the prescribed exposure duration and
the observed flight action time in those tests with
. The flowchart from the input of data set and
to the inference output of
is illustrated in the bottom row of Figure 4.
We construct the formulation for inferring
. Since
is now known (inferred in subsection 3.3), we write the PDF and CDF of the Weibull distribution as parameterized in
with k as the unknown parameter, instead of parameterized in
.
(20)
(21)
When the prescribed exposure duration is
, the time of observed flight action (in those tests with occurrence of flight action) has the conditional density given in (15). The conditional density depends on parameters
, which are the inference variables in our formulation. For clarity, we include
explicitly in the notation of conditional density.
(22)
Given data set
of observed flight action times from those tests with the occurrence of flight action (
), the log-likelihood as a function of
is
(23)
The maximum likelihood estimate for
is
(24)
Once
is available, the standard deviation
is calculated according to (11)
(25)
In (23), the correct domain of the log-likelihood in variable
is
(26)
which is affected by both the observed flight action time
and the prescribed exposure duration
. The maximization of log-likelihood in (24) is subject to constraint (26). Note that constraint (26) is not naturally implied in the analytical expression of
given in (20). Therefore, in a correct numerical implementation of MLE, constraint (26) needs to be enforced explicitly.
It is worthwhile to point out that for a data set of N exposure tests, the summation in (23) contains less than N terms since not all tests of MoCS lead to occurrence of flight action. The fraction of tests with occurrence of flight action (
) varies with the distribution of prescribed exposure duration
. In a sequence of tests designed using the adaptive Bayesian method [11] [17],
is selected sequentially based on the current posterior to maximize the inference accuracy for
given the total number of tests allocated. The sequence
produced by the Bayesian method is a random walk of
toward and then around the true value of median subjective threshold
. With this Bayesian experimental design, the rate of flight action occurrence is approximately 50% (i.e. about 50% of exposure tests lead to
).
4. Numerical Tests
4.1. Generating Artificial Data Sets
Recall that in our model described in section 2, the subjective threshold on exposure duration for flight action (
) is a random variable while the human reaction time (
) is a deterministic quantity. In each individual test based on the method of constant stimuli (MoSC), given the prescribed exposure duration
, the binary outcome regarding the occurrence of flight action (
) and the time of observed flight action if it occurs (
) are completely determined by the realized sample of
in that particular test.
(27)
(28)
An artificial data set
consisting of N exposure tests is generated by drawing N independent samples of random variable
.
In our numerical tests, we examine the performance of the inference method described in section 3 on artificial data sets generated from 6 distribution types for
.
· Weibull distribution
· Log-normal distribution
· Truncated normal distribution
· Gamma distribution
· Triangular distribution
· Uniform distribution
4.2. 6 Distribution Types for tc Used in Data Generation
4.2.1. Weibull Distribution
The Weibull distribution has been described in subsection 3.2. In particular, we established that the mapping from
to
is numerically invertible.
4.2.2. Log-Normal Distribution
The PDF and CDF of the log-normal distribution are
(29)
(30)
(31)
where
and
are respectively the mean and the variance of
. Functions
and
are the PDF and CDF of the standard normal distribution.
(32)
(33)
The standard normal CDF
is expressed in terms of the error function
defined as
The mean, median, and standard deviation of
are
(34)
(35)
The mapping from
to
is invertible. The inverse mapping is
4.2.3. Truncated Normal Distribution
The PDF and CDF of the truncated normal distribution are
(36)
(37)
(38)
where
and
are respectively the mean and the variance of the untruncated normal distribution.
and
are the standard normal PDF and CDF given in (32) and (33).
is the indicator function of interval
defined as
(39)
Quantities Z and
are defined in terms of
and
as
(40)
The mean, median, and standard deviation of
are
(41)
(42)
where
is the inverse function of
, and from (33), it has the expression
(43)
The mapping from
to
is numerically invertible.
4.2.4. Gamma Distribution
The PDF and CDF of the gamma distribution are
(44)
(45)
(46)
where k is the shape parameter and
the scale parameter of the gamma distribution.
is the lower incomplete gamma function defined as
(47)
The mean, median, and standard deviation of
are
(48)
(49)
where
is the inverse function of
with respect to variable s. It satisfies
(50)
The mapping from
to
is numerically invertible.
4.2.5. Triangular Distribution
The PDF and CDF of the triangular distribution are
(51)
(52)
(53)
where a is the lower limit,
the upper limit, and
the mode of the triangular distribution. We consider the case of
(i.e., the distribution is symmetric or right-skewed). The mean, median, and standard deviation of
have the expressions below.
(54)
(55)
The general triangular distribution has 3 parameters
. In order to map
back to parameters, we reduce the number of parameters to 2. We select
as independent parameters and set parameter a as a function of
by enforcing a symmetric triangular distribution or a right-skewed distribution subject to constraint
.
In this subclass of triangular distributions parameterized by
, the mapping from
to
is numerically invertible.
4.2.6. Uniform Distribution
The PDF and CDF of the uniform distribution are
(56)
(57)
(58)
where a is the lower limit and
the upper limit of the uniform distribution. The mean, median, and standard deviation of
are
(59)
(60)
The mapping from
to
is invertible. The inverse mapping is
For
, however, the inverse mapping yields a uniform distribution with
, which is invalid for describing
since the exposure duration and the subjective threshold on the exposure duration are always positive.
4.2.7. Graphs of the 6 Distribution Types
In our numerical simulations below, we test the inference method on data sets generated from the 6 distribution types described above. In each distribution type, we explore 4 cases of different relative distribution widths as measured by
. Since the problem is invariant up to a constant scaling, in the data generation, we fix
and examine the cases of
, 0.5, 1, and 1.5. In the case of
, we use only 5 distributions, excluding the uniform distribution since it gives an invalid distribution protruding to
. The graphs of PDF
are shown in the 6 panels of Figure 5, one panel for each of the 6 distribution types. Each panel contains 4 graphs of
, one for each value of
, except the bottom right panel (the uniform distribution), which contains only 3 graphs.
4.3. Monte Carlo Simulations and Results
4.3.1. Simulation Parameters and Procedures
We generate artificial data sets from the 6 distribution types listed above. In data generation, the true non-dimensional values of
,
and
are set to
This set of non-dimensional values is consistent with the physical situation where the subjective threshold is
, the human reaction time
and the time scale for non-dimensionalization
. It is important to emphasize that these true values are used only in data generation and in evaluating the performance of inference method (by comparing the inferred values to these true values). These true values are unknown in the inference process, which uses only the artificial data sets generated.
As described in our previous study [11], a set of N exposure tests is designed sequentially using the adaptive Bayesian method. Specifically, the exposure duration for the next test
is calculated based on the current posterior of the unknown
(which is viewed as a random variable in the Bayesian framework).
Figure 5. Six distribution types combined with four values of
yield 23 distributions for
.
is invalid for the uniform distribution.
is fixed in all distributions.
Once
is determined and the true distribution for
is selected (out of the 6 distributions), we sample random variable
and generate the binary outcome and the observed flight action time
according to (27) and (28).
The inference of
and
is carried out in two steps (see the schematic illustration in Figure 4). In the inference of
, only the data set of binary outcomes
is used. The subsequent inference of
utilizes the data set of observed flight action times
as well as the already estimated
. Each Monte Carlo run consists of generating one data set of N tests and carrying out inference on the data set to produce one sample of each inferred parameter. Histograms, scatter plots and root-mean-square (RMS) errors of the inferred values are calculated based on
Monte Carlo repeats.
4.3.2. Inference Results of mc
is inferred from data of binary observations
.
We first exhibit in two figures the histograms of inferred
using data sets generated from the 6 distributions with
. Figure 6 shows the results on data from i) Weibull (top row), ii) log-normal (middle row) and iii) truncated normal distribution (bottom row); Figure 7 displays the results on data from iv) gamma (top row), v) triangular (middle row) and vi) uniform distribution (bottom row). Each histogram is based on
Monte Carlo repeats (i.e., based on 1000 samples of inferred
, one sample from each Monte Carlo repeat). In all situations, the inference formulation is based on the Weibull distribution as described in section 3. In Figure 6 and Figure 7, the left column is
(number of exposure tests in the data set of each Monte Carlo repeat), the right column
. A comparison of the two columns indicates that the inferred
converges to the true value as the sample size is increased.
Next, we examine the RMS error of the inferred
, defined below and calculated in simulations based on
Monte Carlo repeats.
(61)
The RMS errors of inferred
vs N are plotted in the 6 panel of Figure 8. Each panel is labeled with the distribution used in data generation, and contains 4 curves corresponding to the 4 values of
used in data generation. For all data sets, the RMS error decreases as the sample size N is increased, and at a fixed N, the RMS error is higher on a data set generated using a larger value of
. That is, when the uncertainty in subjective threshold
is larger in the underlying data, the inference error is higher.
In Figure 9, we compare directly the RMS errors of inferred
vs N on data from the 6 distributions. The inference errors on data from the uniform distribution are somewhat larger than those from other 5 distributions. This is expected since the uniform distribution is the farthest from the inference model (see graphs of PDFs in Figure 5). Figure 8 and Figure 9 demonstrate that the inference of median subjective threshold
based on the Weibull distribution converges to the correct value even when the underlying true model of the data deviates from the inference model.
Figure 6. Histograms of inferred
on data from 1) Weibull (top row), 2) log-normal (middle row) and 3) truncated normal distribution (bottom row). Left column:
; right column:
. Each histogram is based on 1000 Monte Carlo repeats.
Figure 7. Histograms of inferred
on data from 4) gamma (top row), 5) triangular (middle row) and vi) uniform distribution (bottom row). Left column:
; right column:
. Each histogram is based on 1000 Monte Carlo repeats.
Figure 8. The RMS errors of inferred
vs N on data generated from the 6 distributions (one panel for each distribution) with the 4 values of
.
Figure 9. Comparison of RMS errors in inferred
vs N on data from the 6 distributions. Left: results on data with
; right:
.
4.3.3. Inference Results of (mR, sc)
is inferred from data of observed flight action times
. We explore the scatter plots of inferred
using data sets generated from the 6 distributions with
. Figure 10 presents the results on data from 3 of the 6 distributions: 1) Weibull (top row), 2) log-normal (middle row) and 6) truncated normal distribution (bottom row); Figure 11 displays the results on data from the other 3 distributions: 4) gamma (top row), 5) triangular (middle row) and 6) uniform distribution (bottom row).
Each scatter plot is based on
Monte Carlo repeats. In all situations, the inference formulation is based on the Weibull distribution as described in section 3. In Figure 10 and Figure 11, the left column is
, the right column
. Note that N is the number of exposure tests in the data set of each Monte Carlo repeat. In the Bayesian experimental design intended for optimizing the inference of
, about half of tests produce no flight action. Only those tests with the occurrence of flight action are relevant in the inference of
. As a result, the effective sample size for inferring
is about N/2. A comparison of the two columns indicates that as the sample size is increased, the inferred
(the human reaction time) converges to the true value while the inferred
(standard deviation of the subjective threshold) converges to an incorrect value. For example, for data from the log-normal distribution, the inferred
converges to a value lower than the true
.
To investigate the convergence of inferred
, we plot RMS errors of inferred
vs N in the 6 panels of Figure 12. Each panel is labeled with the distribution used in data generation, and contains 4 curves corresponding to the 4 values of
used in data generation. For all data sets, the RMS error of inferred
decreases as the sample size N is increased, and at a fixed N, the RMS error of inferred
is higher on a data set generated using a larger value of
. That is, when the uncertainty in subjective threshold
is larger in the
Figure 10. Scatter plots of inferred
on data from 1) Weibull (top row), 2) log-normal (middle row) and 3) truncated normal distribution (bottom row). Left column:
; right column:
. Each scatter plot is based on 1000 Monte Carlo repeats.
Figure 11. Scatter plots of inferred
on data from 4) gamma (top row), 5) triangular (middle row) and 6) uniform distribution (bottom row). Left column:
; right column:
. Each scatter plot is based on 1000 Monte Carlo repeats.
Figure 12. The RMS errors of inferred
vs N on data generated from the 6 distributions (one panel for each distribution) with the 4 values of
.
Figure 13. Comparison of RMS errors in inferred
vs N on data from the 6 distributions. Left: results on data with
; right:
.
underlying data, the inference error of human reaction time
is higher.
In Figure 13, we compare directly the RMS errors of inferred
vs N on data from the 6 distributions. The inference errors are comparable on data from all 6 distributions, with data from the uniform distribution yielding slightly larger inference errors. Figure 12 and Figure 13 demonstrate that the inference of human reaction time
based on the Weibull distribution converges to the correct value even when the underlying true model of the data deviates from the inference model.
5. Concluding Remarks
In this study, we consider the psychophysical experiments in which a test subject is exposed to a stimulus source for a prescribed duration and the test subject’s response is binary. For example, the occurrence of flight action is binary when a subject is exposed to a millimeter wave beam for a prescribed duration. The binary outcome is determined by the prescribed exposure duration and a subjective threshold of the test subject on exposure duration. To include the biovariability, we view the subjective threshold as a random variable. In an exposure test, if the realized sample value of subjective threshold is lower than the prescribed duration, flight action is initiated at the time of subjective threshold, which is not measurable. The actuation of flight action is observed at a later time, which is recorded in experiments. The delay from the initiation time to the actuation time of flight action is the human reaction time, which is not measurable. Here, we view the human reaction time as an unknown deterministic quantity. The case of a random variable human reaction time will be investigated in a subsequent study. We consider the experiments based on the method of constant stimuli (MoCS) in which the exposure duration is prescribed before each test. This is different from the experiments based on the method of limits (MoL) in which the beam stimulus is kept on until the observed actuation of flight action. In a test of MoL, flight action always occurs; in the observed actuation time of flight action, the subjective threshold and the human reaction are tangled together, making it difficult to infer either one. In contrast, in a test based on MoCS, flight action may or may not occur; the observed binary outcome is independent of human reaction time and it provides a binary description of the realized sample of subjective threshold in that test, making it possible to estimate the median subjective threshold from a sequence of properly designed tests.
In our modeling framework, the human reaction time and the distribution of random subjective threshold are unknown. The primary objective of the current study is to estimate the human reaction time from data of prescribed exposure durations, binary outcomes and actuation times of flight action (if it occurs), measured in a sequence of tests. We model the random subjective threshold
as a Weibull distribution. We estimate the human reaction time
from data in two inference steps. In step one, we use only the data of binary outcomes, which excludes the effect of human reaction time. We fix the shape parameter of Weibull distribution at
and use the maximum likelihood estimation (MLE) to infer the median subjective threshold
from data of binary outcomes. In method of constant stimuli, a critical part of experimental design is to specify the exposure duration in each test. In our previous study [11], we used an adaptive Bayesian method to select the exposure duration for the next test based on the current posterior of the unknown median
. This experimental design produces the optimal inference on the median subjective threshold
and has been used in real exposure tests. In the current study, we adopt this experimental design. Once
is estimated, in step two of the inference, we estimate the human reaction time
from data of observed actuation times of flight action. We parameterize the Weibull distribution by the known median
and the unknown standard deviation
; we use the maximum likelihood estimation (MLE) to infer the two unknowns:
and
, both of which affect the distribution of observed actuation times of flight action.
We run Monte Carlo simulations to test the performance of the proposed inference method. Recall that the inference formulation is based on the Weibull distribution. To explore its robustness, we test the inference method on data generated from 6 different distribution types as the underlying true model of random subjective threshold. To assess the effect of randomness of the subjective threshold, we test the inference method on data generated with 4 values of relative distribution width of subjective threshold:
in each model. In each case, to study the convergence of the inference, we run the inference method on data with sample size ranging from
to
. For each set of (model distribution, simulation parameters), we carry out 1000 Monte Carlo repeats to generate 1000 samples of each estimated parameter. The convergence is examined by plotting the RMS inference error vs sample size (N). Based on the results of extensive numerical tests, we draw several comments, observations and conclusions on the inference method.
· The inference of median subjective threshold (
) uses only the data of binary outcomes in the method of constant stimuli. The inferred
converges to the correct value as the sample size increases, even when the underlying true model of random subjective threshold deviates from the inference model (Weibull distribution).
· The inference of human reaction time (
) utilizes the data of observed actuation times of flight action (if it occurs). The inferred
converges to the correct value as the sample size increases, even when the underlying true model of random subjective threshold deviates from the inference model (Weibull distribution).
· The standard deviation of subjective threshold (
) is a by-product in the inference of
. However, when the underlying true model of random subjective threshold deviates from the inference model, the inferred
converges to an incorrect value.
· The robustness of inferring both the median subjective threshold
and the human reaction time
makes the inference method applicable to real experimental data in which the underlying true model of the random subject threshold is unknown. In contrast, the inference of standard deviation of subjective threshold (
) depends on knowing the underlying true model, which makes it impractical.
· The inference errors of
and
decrease as the standard deviation of subjective threshold
is reduced. That is, for a hypothetical system with a lower uncertainty in the subjective threshold, the inference errors would be smaller.
The inference discussed above utilizes the data collected in a sequence of exposure tests designed with the objective of optimizing the inference of only the median subjective threshold
. We extend the usage of this same data set to infer the human reaction time
in a two-step procedure. This approach is practical and operationally important since data sets of this type already exist from real experiments. Now suppose we revise the experimental design with the objective of optimizing the inference of both
and
. Will the resulting sequence of tests be significantly different from the current one in terms of the distribution of prescribed exposure durations? Will the new experimental design notably improve the inference accuracy of
and
over the current one? Will the inference accuracy improve if we infer
and
simultaneously instead of in a two-step procedure? These questions will be explored in a future study.
Acknowledgements and Disclaimer
The authors acknowledge the Joint Intermediate Force Capabilities Office of U.S. Department of Defense and the Naval Postgraduate School for supporting this work. The views expressed in this document are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government.