Upload
colin-wg-clifford
View
216
Download
3
Embed Size (px)
Citation preview
Getting technical about awarenessColin W.G. Clifford, Ehsan Arabzadeh and Justin A. Harris
School of Psychology, The University of Sydney, Sydney, NSW 2006, Australia
Opinion
It has recently been argued that post-decision wageringprovides an objective measure of awareness. We criti-cally evaluate this claim, emphasizing the distinctionbetween performance without awareness and a reluc-tance to gamble in full awareness of weak sensoryevidence. We address two key methodological issues.The first is the design of the pay-off matrix to reward astrategy of wagering that reflects the strength of sensoryevidence. The second is the use of signal detectiontheory to analyze the resulting data. We argue thatproper treatment of these issues is essential if post-decision wagering is to prove valuable in validatingclaims of perception without awareness in normal sub-jects and neuropsychological patients.
Measuring awarenessHow activity in our brains gives rise to conscious aware-ness is a fundamental question in cognitive science [1]. Butwhat is awareness, and how can we assess whether or notsomeone is aware of something? Researchers haveattempted to answer these questions by investigatingthe strength of confidence that subjects have in theirdecisions. Following presentation of a stimulus, subjectsare typically required to make two responses: a psycho-physical judgment about some attribute of the stimulusand a rating of their confidence in that judgment [2,3].However, techniques using post-decision confidence rat-ings have been criticized on methodological grounds [4–6].A recent report by Persaud and colleagues argues that anobjective measure of awareness is provided instead by thetechnique of post-decision wagering [7]. Following a psy-chophysical decision, subjects are required to bet on itscorrectness by making either a high or a low wager. If thedecision was correct, the subject wins the amount wagered;otherwise they lose that amount. Subjects thus have theopportunity to convert their psychophysical performanceinto financial profit.
Post-decision wageringThe potential novelty of the approach proposed by Persaudand colleagues lies in the use of wagers rather than sub-jective confidencemeasures. The rationale for usingwager-ing is that it ‘exploits people’s desire to make money’ [8]and that ‘participants find post-decision wagering to bemore intuitive than were subjective measures’ [7]. Persaudand colleagues claimed that failure to maximize earningson a post-decision wager demonstrates that decisions aremade without awareness. Here, we challenge this claim ontheoretical grounds using signal detection theory [9].We demonstrate, through the behavior of a hypothetical
Corresponding author: Clifford, C.W.G. ([email protected]).
1364-6613/$ – see front matter � 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2007.
observer, that the previously proposed analysis ofpost-decision wagering fails to distinguish between per-formancewithout awareness and a reluctance to gamble onthe basis of full awareness of weak sensory evidence. Toremedy this flaw, we propose two fundamental changesthat are essential if post-decision wagering is to provide anobjective measure of awareness.
We focus here on the first of the three tasks described byPersaud and colleagues, involving decisions about thepresence or absence of a stimulus that was in fact presentin 50% of trials. The data we have reanalyzed come frompatient GY, who consistently shows blindsight, a paradox-ical dissociation between discrimination and awareness ofvisual stimuli following damage to striate cortex (see, e.g.,Ref. [10]). We do not wish to claim that GY does not exhibitblindsight, although it is interesting to note that the Yes–No task with which Persaud and colleagues claim todemonstrate perception without awareness in GY is pre-cisely the type of judgment on which GY has previouslybeen shown to be impaired relative to his performance on aforced-choice task [11]. What we are disputing is the claimby Persaud and colleagues that the analysis of post-decision wagering ‘measures awareness directly’.
Consider the summary data of Persaud and colleaguespresented in Table 1. It is evident that their subject did notconsistently wager high after correct decisions: of 141correct trials, he wagered high on only 48% (67/141). Hethus failed to earn as much as if he had wagered high afterall correct trials. On the basis of this failure to maximizewinnings, Persaud and colleagues concluded that thesedata demonstrate perception without awareness. If thatconclusion was logically sound, we reasoned, it should beimpossible for the reported pattern of results to have arisenfrom a situation in which GY based the magnitude of hispost-decision wager on knowledge of the same sensoryevidence upon which the Yes–No decision had been based.
Testing awareness with a hypothetical observerWe designed a hypothetical observer using the followingsimple strategy for making each decision and subsequentwager. For the decision, if the weight of sensory evidenceexceedsa certain threshold then the observer responds ‘Yes’,otherwise ‘No’. For the wager, the observer bets high follow-ing a ‘Yes’ only if the weight of sensory evidence exceeds agreater (more conservative) criterion level. Following a ‘No’,the observer bets high only if the weight of sensory evidencefails even to reach a lower (more liberal) criterion. The Yes–No decision and the subsequent wager are thus based uponknowledge of the same sensory evidence.
For ease of exposition we will consider a single case inwhich the observer is much more likely to wager highon ‘Yes’ decisions than on ‘No’ decisions, reflecting the
11.009 54
Table 1. Number of high and low wagers made by subject GY asa function of the correctness of the preceding detectionjudgment
Correct Incorrect Total wagers
High wager 67 23 90
Low wager 74 36 110
Total responses 141 59 200
Data reproduced from Ref. [7].
Box 1. Signal detection theory
Signal detection theory assumes that, in a Yes–No detection task, an
observer will respond positively when the sensory evidence for the
presence of the target stimulus exceeds a criterion level set by the
observer. For a target stimulus of given strength, the responses of the
observer can be described by a single operating characteristic
specifying the degree of overlap between the signal-plus-noise and
noise-only distributions (Figure Ia). Signal detection theory enables
us to quantify the subjective strength of the stimulus according to the
metric d0 [9], which is computed from the hit rate and false alarm rate
of the observer. The hit rate is the proportion of target-present trials
on which the observer responds ‘Yes’; the false alarm rate is the
proportion of target-absent trials on which the observer responds
‘Yes’. The subjective strength of the stimulus determines the shape of
Figure I. Subjective signal strength, bias and accuracy. (a) Subjective signal strengt
evidence on target-present and target-absent trials. (b) For a given d0, the hit rate and fa
a point determined by the decision criterion of the observer. Each of the operating c
deviation from the major diagonal. (c) The decision criterion quantifies the weight of s
Each of the isobias curves illustrated corresponds to a single criterion (as applied ac
proportion of correct responses is the average of the hit rate and the correct rejection r
to a particular accuracy; accuracy increases with deviation from the major diagonal. (
criterion. Thus, the same accuracy can result from different subjective signal strength
from the same subjective signal strength.
Opinion Trends in Cognitive Sciences Vol.12 No.2
plausible bias of an observer to place more weight onthe presence than absence of evidence. We also makethe assumption, conventional in signal detection theory(see Box 1), that the weight of sensory evidence for thepresence of the target signal is given by the strength ofthe signal corrupted by additive Gaussian noise, whereasthe evidence on target-absent trials is given simply by the
the operating characteristic (Figure Ib), whereas variations in criterion
level determine the location on that operating characteristic (Figure
Ic).
The accuracy of responding of a subject, in terms of the overall
proportion of correct responses, depends not only on the subjective
signal strength but also on the response criterion (Figure Id,e). Thus, it
is unsafe to make inferences about subjective signal strength on the
basis of accuracy. Post-decision wagering rewards accuracy, because
it does not discriminate between hits and correct rejections (correct
decisions) or between misses and false alarms (incorrect decisions).
Thus, it is unsafe to make inferences about the availability of sensory
evidence to awareness purely on the basis of the accuracy of post-
decision wagering.
h, d0, is a measure of the separation of the probability distributions of sensory
lse alarm rate of the observer lie on the corresponding operating characteristic at
haracteristics illustrated corresponds to a particular d0; sensitivity increases with
ensory evidence that the observer requires to report that the target was present.
ross the sensitivity continuum). (d) The accuracy of an observer in terms of the
ate (1 – false alarm rate). Each of the isoperformance lines illustrated corresponds
e) Accuracy depends not only on subjective signal strength but also on response
s, depending on the response criterion. Similarly, different accuracies can result
55
Figure 1. Different criteria for responding and wagering. Signal-plus-noise (red) and noise-only (black) probability distributions of sensory evidence on target-present and
target-absent trials, respectively. The degree of overlap between these distributions determines the operating characteristic of the observer, quantified in terms of d0.
Vertical lines denote decision (green) and wagering (blue) criteria for a hypothetical observer (Table 2) designed to reproduce the performance levels reported by Persaud
and colleagues (Table 1). Each wager is based upon the same sensory evidence as the preceding decision and differs only in criterion level. Units along the horizontal axis
are standard deviations of the noise distribution.
Opinion Trends in Cognitive Sciences Vol.12 No.2
same Gaussian noise distribution (Figure 1). It isimportant to note that our conclusions do not depend oneither of these assumptions, which in fact serve only tolimit the range of possible scenarios that might give rise tothe pattern of results reported by Persaud and colleagues.
Table 2 shows the responses of our hypothetical obser-ver to 200 psychophysical trials, half of which contain thetarget signal and half of which do not. For the Yes–Nodecision, the hit rate is the proportion of target-presenttrials on which the observer correctly reports the target,and the false alarm rate is the proportion of target-absenttrials on which the observer erroneously responds ‘yes’.Together, the hit rate and false alarm rate provide ameasure of subjective signal strength, d0, the separationof the probability distributions of sensory evidence ontarget-present and target-absent trials (see Box 1). Forour hypothetical observer, the hit rate is 75% (63 highwagers and 12 low wagers following a ‘Yes’ on 100 target-present trials) and the false alarm rate is 34% (23 highwagers and 11 low wagers following a ‘Yes’ on 100 target-absent trials), corresponding to a d0 of 1.09. We can treatthe boundary between low and high wagers following a‘Yes’ response in the same way. In this case the hit rate is63% (63 highwagers following a ‘Yes’ on 100 target-presenttrials) and the false alarm rate is 23% (23 high wagersfollowing a ‘Yes’ on 100 target-absent trials), correspondingto a d0 of 1.07. Thus, it can be seen that our hypotheticalobserver is basing the decision and subsequent wager onthe same weight of sensory evidence (d0 = 1.08 � 0.01), theonly difference being the more conservative criterion forwagering high than for responding ‘Yes’. The boundarybetween low and high wagers following a ‘No’ is also
Table 2. Responses of a hypothetical observer
Decision No Yes Total
Wager High Low Low High
Target present 0 25 12 63 100
Target absent 4 62 11 23 100
56
consistent with a d0 of 1.08: for a correct rejection rate of4% the expected number of misses on 100 target-presenttrials would be zero (0.23%).
If we express these data in the form used for the analysisproposed by Persaud and colleagues [7] then we record 67(4 + 63) high wagers following correct responses, 23(0 + 23) high wagers following incorrect responses, 74(62 + 12) low wagers following correct responses, and 36(25 + 11) low wagers following incorrect responses. This isexactly the pattern of responding from which they con-cluded that GY was not always aware that he was makingcorrect decisions. That our hypothetical observer performsin exactly this way demonstrates that the dissociationbetween performance and wagering reported by Persaudand colleagues does not necessarily indicate a lack ofawareness of the sensory evidence upon which perform-ance is based (see Box 2).
Interpretation of post-decision judgmentsThe optimal strategy for wagering in the experimentsdescribed by Persaud and colleagues is actually to wagerhigh regardless of the weight of the sensory evidence. Thisis because equal amounts stand to be won and lost on anygiven wager. If the observer is performing at chance then itdoes not matter whether they bet high or low, theirexpected gain is zero. However, the expected gain forany performance above chance is proportional to theamount wagered. Thus, the best strategy of the observerfor maximizing expected gain is always to wager highregardless of the weight of the sensory evidence. This begsthe question: how can a failure of a subject to wageroptimally be a measure of lack of awareness of the sensoryevidence when the optimal strategy is independent of thatevidence? Instead, it demonstrates that the subject lacksawareness of the optimal strategy for wagering. This ques-tions the rationale for using wagering instead of confidencemeasures because it suggests that wagering might notbe as intuitive as its proponents claim. Of course, if the
Box 2. Comparison between the blind and normal visual
fields of subject GY
Persaud and colleagues dismiss the possibility that failure of GY to
optimize his winnings could reflect a reluctance to gamble in full
awareness of weak sensory evidence. They claim instead that this is
an instance of perception without awareness, and that his wagering
follows a qualitatively different pattern when he is aware of the
stimuli. For a comparison between wagering in the presence and
absence of awareness to be meaningful, it is essential that the
subjective strength of the sensory evidence be equated between the
two conditions. GY has a unilateral lesion in his left striate cortex,
resulting in a unilateral visual deficit, right homonymous hemi-
anopia. It has previously been demonstrated that the residual vision
in the hemianopia of GY is not like normal, near-threshold vision,
but instead shows a dissociation between performance and
awareness [11]. An appropriate way to equate the subjective
strength of sensory evidence in the presence and absence of
awareness would thus be to equate performance (d0) in the normal
and blind visual fields by using a low-contrast stimulus near to
detection threshold in the normal visual field. However, Persaud and
colleagues chose instead to equalize overall performance across a
block of trials by manipulating the proportion of intermixed
extremely easy (96% stimulus contrast) and extremely hard (1%
stimulus contrast) trials. Given that wagers are made on a trial-by-
trial basis, the overall performance level across a block of
intermixed extremely easy and extremely hard stimuli is not a
suitable metric to equate with performance on the original
experiment. On no trial in the control experiment was the subjective
strength of the sensory evidence equal to that in the original
experiment, where sensory evidence was consistently present but
weak. This failure to equate subjective signal strength in a
satisfactory manner undermines the argument by Persaud and
colleagues that the wagering of GY follows qualitatively different
patterns depending on whether or not he is aware of the stimuli.
Instead, their results are entirely consistent with a wagering strategy
whereby the subject makes a high wager only when the sensory
evidence exceeds a certain threshold.
Table 3. Example pay-off matrix designed to make the optimalstrategy for post-decision gambling a function of the weight ofsensory evidence
Outcome of decision
Correct Incorrect
Wager Low +2 �1
High +5 �5
Opinion Trends in Cognitive Sciences Vol.12 No.2
observer believes they have made the wrong decisionthen they should wager low. But this only begs anotherquestion: why would an observer make a decision thatthey believe is wrong when they only win on correctdecisions?
Figure 2. Distinguishing wagers made with and without awareness. (a) The intersection
the operating characteristic of the observer (unbroken line) define the optimal strateg
decision boundary producing optimal accuracy on the Yes-No task. (b) Wagers made w
based lie on the sensory operating characteristic even if the strategy for wagering is sub
defined by the ratio of hits to false alarms following a ‘Yes’ decision and on a line def
interest then becomes whether the resulting data plotted as hit rate versus false alarm ra
(1, 1), respectively, or by a conventional receiver operating characteristic (ROC) curve. A
being carried out without full access to the sensory evidence upon which the original
Making the optimal wagering strategy depend on thesensory evidenceIf post-decision wagering is to prove useful as a measure ofawareness, it is essential to make the optimal strategy forgambling a function of the weight of sensory evidence. Toreward a strategy of wagering low when the sensory evi-dence is weak but wagering high if the evidence is strong,the ratio of potential gain-to-loss should be greater for lowwagers than for high. This can be achieved by employing apay-off matrix such as the one illustrated in Table 3.
In this specific example, when perceptual performanceis at chance (probability that the response is correct = 50%)the expected return from a low wager is +1/2 [= (+2 � 1)/2]comparedwith 0 from a highwager [=(+5 � 5)/2]. But whenperceptual performance is perfect (probability that theresponse is correct = 100%) a low wager returns only +2compared with +5 from a high wager. The optimal wager-ing strategy is determined by the differential loss of wager-ing incorrectly (5 � 1 = 4) and the differential gain ofwagering correctly (5 � 2 = 3) between high and lowwagers. Specifically, the optimal strategy is to wager lowon trials where the weight of sensory evidence is such thatthe probability of a correct response is less than thedifferential loss of wagering incorrectly divided by thesum of the differential loss and the differential gain. Inthis example, the optimal strategy is to wager low on trialswhere the probability of a correct response is less than four-sevenths and to wager high with stronger evidence. Whenthe probability of a correct response is exactly four-sevenths it can be confirmed that the expected return fromlow and high wagers is equal (at +5/7).
s of the optimal boundary between high and low wagers (curved dotted lines) and
y of the observer for post-decision wagering (red dots). The blue dot marks the
ith full awareness of the sensory evidence upon which the original decision was
optimal. (c) Wagers made with no awareness of the sensory evidence lie on a line
ined by the ratio of misses to correct rejections following a ‘No’. The question of
te are significantly better fitted by a pair of straight lines passing through (0, 0) and
significant result would support the hypothesis that post-decision wagering was
decision had been made. Red arrows indicate departures from optimal wagering.
57
Opinion Trends in Cognitive Sciences Vol.12 No.2
Distinguishing suboptimal wagering from lack ofawarenessThe example shows that even a simple pay-off matrix canlead to an optimal wagering strategy that would be hard fora subject to operationalize. This highlights that a depar-ture from advantageous wagering does not necessarilyindicate lack of awareness of the sensory evidence. Here,we propose an analysis of the wagering data that avoidsconfounding lack of awareness of the optimal wageringstrategy with lack of awareness of the weight of sensoryevidence. The key point is that post-decision wagers madewith knowledge of the sensory evidence should still lie onthe same operating characteristic as the original decisioneven if the wagering strategy is suboptimal (Figure 2). Bycontrast, post-decision wagers made without awareness ofthe sensory evidence should be made on the same pro-portion of hits-to-misses as the original decision. In thelatter case, the wagering of the subject would be as thoughthey were randomly wagering high after a certain pro-portion of ‘Yes’ decisions and a certain (possibly different)proportion of ‘No’ decisions irrespective of the weight ofsensory evidence upon which the decision had been based.
Concluding remarksWe have argued that if post-decision wagering is to provevaluable then proper treatment of two specific issues isessential. The first is the design of the pay-off matrix,which must ensure that the optimal strategy for wageringis a function of the weight of sensory evidence. The secondis the analysis of the resulting data, which must have thecapacity to distinguish the use of a suboptimal wageringstrategy from a genuine lack of awareness of the sensory
The ScienceDire
ScienceDirect’s extensive and unique full-text colle
titles such as The Lancet, Cell, Tetrahedron and the
Discovery Today journals. With ScienceDirect, the r
searching and linking functionality, a
The rapid growth of the ScienceDirect collection is
publications and the ongoing addition to the Ba
disciplines. The latest step in this ambitious proje
volume one, issue one, is the addition of the h
ScienceDirect. Also available online for the first t
containing more than 12,000 articles that highlight
life scien
For more information, visit
58
evidence. We have shown here how the original techniqueof post-decision wagering as proposed by Persaud andcolleagues [7] can be modified to fulfill these criteria. Itremains to be tested empirically whether there is anadvantage in using post-decision wagering rather thanconfidence judgments to validate claims of perceptionwith-out awareness.
AcknowledgementsThis work is supported by Discovery Project Grants to C.C. and J.H. fromthe Australian Research Council and a Human Frontiers ScienceProgramme Long-Term Fellowship to E.A.
References1 Koch, C. (2004) The Quest for Consciousness; a Neurobiological
Approach, Roberts and Company Publishers2 Kolb, F.C. and Braun, J. (1995) Blindsight in normal observers.Nature
377, 336–3383 Kunimoto, C. et al. (2001) Confidence and accuracy of near-threshold
discrimination responses. Conscious. Cogn. 10, 294–3404 Morgan, M.J. et al. (1997) Blindsight in normal subjects? Nature 385,
401–4025 Galvin, S.J. et al. (2003) Type 2 tasks in the theory of signal
detectability: discrimination between correct and incorrect decisions.Psychon. Bull. Rev. 10, 843–876
6 Evans, S. and Azzopardi, P. (2007) Evaluation of a ‘bias-free’ measureof awareness. Spat. Vis. 20, 61–77
7 Persaud, N. et al. (2007) Post-decision wagering objectively measuresawareness. Nat. Neurosci. 10, 257–261
8 Koch, C. and Preuschoff, K. (2007) Betting the house on consciousness.Nat. Neurosci. 10, 140–141
9 Green, D.M. and Swets, J.A. (1966) Signal Detection Theory andPsychophysics, Wiley
10 Stoerig, P. et al. (2002) Aware or unaware: assessment of corticalblindness in four men and a monkey. Cereb. Cortex 12, 565–574
11 Azzopardi, P. and Cowey, A. (1997) Is blindsight like normal, near-threshold vision? Proc. Natl. Acad. Sci. U. S. A. 94, 14190–14194
ct collection
ction covers more than 1900 journals, including
full suite of Trends, Current Opinion and Drug
esearch process is enhanced with unsurpassed
ll on a single, intuitive interface.
a result of the integration of several prestigious
ckfiles - heritage collections in a number of
ct to digitize all of Elsevier’s journals back to
ighly cited Cell Press journal collection on
ime are six Cell titles’ long-awaited Backfiles,
important historic developments in the field of
ces.
www.sciencedirect.com