User Settings of Cue Thresholds for Binary Categorization Decisions

The output of binary cuing systems, such as alerts or alarms, depends on the threshold setting a parameter that is often user-adjustable. However, it is unknown if users are able to adequately adjust thresholds and what information may help them to do so. Two experiments tested threshold settings for a binary classification task based on binary cues. During the task, participants decided whether a product was intact or faulty. Experimental conditions differed in the information participants received: all participants were informed about a product's fault probability and the payoffs associated with decision outcomes; one third also received information regarding conditional probabilities for a fault when the system indicated or did not indicate the existence of one (predictive values); and another third received information about conditional probabilities for the system indicating a fault, in the instance of the existence or lack thereof, of an actual fault (diagnostic values). Threshold settings in all experimental groups were nonoptimal, with settings closest to the optimum with predictive-values information. Results corresponded with a model describing threshold settings as a function of the conditional probabilities for the different outcomes. From a practical perspective, results indicate that predictive-values information best supports decisions about threshold settings. Consequently, for users to adjust thresholds, they should receive information about predictive-values, provided that such values can be computed.