Soft-tissue detectability in cone-beam CT: Evaluation by 2AFC testsin relation to physical performance metrics

D. J. Tward and J. H. Siewerdsena�

Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada M5G 2M9,Ontario Cancer Institute, Princess Margaret Hospital, Toronto, Ontario, Canada M5G 2M9

M. J. DalyOntario Cancer Institute, Princess Margaret Hospital, Toronto, Ontario, Canada M5G 2M9

S. RichardDepartment of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada M5G 2M9

D. J. Moseley and D. A. JaffrayOntario Cancer Institute, Princess Margaret Hospital, Toronto, Ontario, Canada M5G 2M9,Department of Radiation Oncology, University of Toronto, Toronto, Ontario, Canada M5S 3E2,and Radiation Medicine Program, Princess Margaret Hospital, Toronto, Ontario, Canada M5G 2M9

N. S. PaulDepartment of Medical Imaging, University Health Network, Toronto, Ontario, Canada M5G 2M9

�Received 22 November 2006; revised 21 August 2007; accepted for publication 31 August 2007;published 26 October 2007�

Soft-tissue detectability in cone-beam computed tomography �CBCT� was evaluated via two-alternative forced-choice �2AFC� tests. Investigations included the dependence of detectability onradiation dose, the influence of the asymmetric three-dimensional �3D� noise-power spectrum�NPS� in axial and sagittal or coronal planes, and the effect of prior knowledge on detectability.Custom-built phantoms ��15 cm diameter cylinders� containing soft-tissue-simulating spheres ofvariable contrast and diameter were imaged on an experimental CBCT bench. The proportion ofcorrect responses �Pcorr� in 2AFC tests was analyzed as a figure of merit, ideally equal to the areaunder the receiver operating characteristic curve. Pcorr was evaluated as a function of the spherediameter �1.6–12.7 mm�, contrast �20–165 HU�, dose �1–7 mGy�, plane of visualization �axial/sagittal�, apodization filter �Hanning and Ram-Lak�, and prior knowledge provided to the observer�ranging from stimulus known exactly �SKE� to stimulus unknown �SUK��. Detectability limitswere characterized in terms of the dose required to achieve a given level of Pcorr �e.g., 70%�. Forexample, a 20 HU stimulus of diameter down to �6 mm was detected with Pcorr 70% at dose�2 mGy. Detectability tended to be greater in axial than in sagittal planes, an effect amplified bysharper apodization filters in a manner consistent with 3D NPS asymmetry. Prior knowledge had amarked influence on detectability—e.g., Pcorr for a �6 mm�20 HU� sphere was �55% –65% underSUK conditions, compared to �70% –85% for SKE conditions. Human observer tests suggestpractical implications for implementation of CBCT: �i� Detectability limits help to define minimum-dose imaging techniques for specific imaging tasks; �ii� detectability of a given structure can varybetween axial and sagittal/coronal planes, owing to the spatial-frequency content of the 3D NPS inrelation to the imaging task; and �iii� performance under SKE conditions �e.g., image guidancetasks in which lesion characteristics are known� is maintained at a lower dose than in SUK condi-tions �e.g., diagnostic tasks in which lesion characteristics are unknown�. © 2007 American Asso-ciation of Physicists in Medicine. �DOI: 10.1118/1.2790586�

Key words: cone-beam CT, flat-panel detector, imaging performance, detectability, alternativeforced-choice test, 2AFC, noise-power spectrum, image-guided interventions, image-guided radia-tion therapy

I. INTRODUCTION

Cone-beam computed tomography �CBCT� using flat-paneldetectors �FPDs� is under investigation in a broad range ofapplications, including preclinical �e.g., small animalimaging�,1 screening and diagnosis �e.g., breast imaging�,2,3

and image guidance �e.g., surgery4–10 and radiotherapy�.1–14

The technology offers submillimeter spatial resolution and

soft-tissue visibility across a large field of view from a single

4459 Med. Phys. 34 „11…, November 2007 0094-2405/2007/34„

rotation about the object. While challenges to image qualityposed by detector performance and high levels of x-ray scat-ter remain important areas of research, CBCT is being usedin a variety of clinical research studies, and the importanceof maximizing image quality and minimizing radiation doseis clear.

Imaging performance may be quantitatively characterizedin the context of such applications in terms of physical per-

445911…/4459/13/$23.00 © 2007 Am. Assoc. Phys. Med.

4460 Tward et al.: Soft-tissue detectability in cone-beam CT 4460

formance metrics involving the contrast, noise, and spatialresolution in CBCT reconstructions. For example, simpleperformance metrics such as the contrast-to-noise ratio orsignal-difference-to-noise ratio �SDNR�, demonstrate the im-portance of x-ray scatter management to maintain CBCTcontrast. The spatial-frequency-dependent signal and noisecharacteristics are prevalently characterized in terms ofFourier-based metrics such as modulation transfer function�MTF�, noise-power spectrum �NPS�, and noise-equivalentquanta �NEQ�. Analysis of MTF, for example, characterizesthe degree to which three-dimensional �3D� spatial resolutionis isotropic and isoplanatic and helps guide the selection ofFPD design, imaging geometry, and choice of reconstructionfilters.15–18 Similarly, measurement and modeling of the 3DNPS �Refs. 19–22� reveal that the axial plane exhibits a ramp�apodization characteristic �as in conventional CT�.23–25

while the longitudinal NPS is governed by the �aliased� de-tector MTF. The corresponding asymmetry is reflected in the3D NEQ and suggests differences in detectability index as-sociated with axial versus sagittal/coronal planes of visual-ization.

CBCT image quality evaluation by rigorous observerstudies is likely important to eventual clinical deployment.Such studies require careful planning, with attention to nu-merous factors that affect statistical accuracy and bias—e.g.,the number of cases, observers, and repeats, as well as selec-tion of training sets, control of reading conditions, and thedefinition of “truth”. Tests range from relatively simple pref-erence tests �in which an observer selects which image ispreferred in accomplishing a given task� and diagnostic sat-isfaction tests26 �in which an observer rates an image on anordinal scale with respect to a given task� to two- andmultiple-alternative forced-choice tests27–30 �2AFC andMAFC tests, respectively� and receiver operating character-istic �ROC� tests.31–33 Such an evaluation is important tomore widespread clinical use and presents an opportunity tobridge the gap between observer-independent physical met-rics �such as NEQ and detectability index�34 and the perfor-mance of real observers.

This article reports an investigation of soft-tissue detect-ability in CBCT as measured in 2AFC tests as a function of

FIG. 1. �a� Experimental imaging bench for CBCT. �b� Imaging phantom cosagittal planes.

several experimental variables, including: �1� Radiation

Medical Physics, Vol. 34, No. 11, November 2007

dose, �2� size and contrast of the object to be detected, �3�filter selection in axial and sagittal/coronal planes, and �4�prior knowledge provided to the observer. The performancemeasure in such 2AFC tests is Pcorr, the proportion of correctresponses, ranging from 0.5 to 1.0 and related to the area�Az� under the receiver operating characteristic �ROC� curveas described by Green and Swets.33 It is broadly appreciatedthat 2AFC tests are less efficient and require a greater num-ber of images than ROC/LROC/FROC tests to achieve agiven level of precision in Az; moreover, 2AFC tests do notallow characterization of tradeoffs between sensitivity andspecificity. However, 2AFC tests are often conducted in fa-vor of these alternatives29,35–37 because they are well suitedto experiments involving simple phantom or simulated im-ages �e.g., sphere on a uniform background�, place minimaldemands on observers �i.e., do not require a qualitative as-sessment of confidence on each decision�, and are thereforefaster and simpler to conduct in studies involving numerousexperimental variables. Three main studies were performed.The first investigated detectability as a function of dose, ob-ject size, and object contrast. The second probed the differ-ence in detectability of a given object as visualized in axialversus sagittal/coronal planes, with differences in detectabil-ity compared to differences in 3D NPS correlation. The thirdexamined the effect of prior knowledge on low-dose detect-ability limits, with conditions constructed in analogy to di-agnostic imaging �in which the observer has little or no in-formation regarding the object� and image-guidedprocedures �in which the observer may have full knowledgeof the object characteristics�.

II. MATERIALS AND METHODS

II.A. CBCT image requisition and reconstruction

II.A.1. CBCT imaging bench

Images were acquired on the experimental CBCT benchshown in Fig. 1. The bench included an x-ray tube �Rad94 inSapphire housing; W target; 0.4–0.8 mm focal spot; 14° an-ode; Varian Medical Systems, Salt Lake City, UT� powered

ng of a uniform cylinder containing an arrangement of spheres in axial and

nsisti

by a constant potential generator �CPX-380, EMD

4461 Tward et al.: Soft-tissue detectability in cone-beam CT 4461

Inc., �Montreal, QC��. The flat-panel detector �RID-1640A,PerkinElmer Optoelectronics, �Santa Clara, CA�� was basedon a 1024�1024 �41�41 cm2� active matrix of a-Si:Hphotodiodes and thin-film transistors with a 400 �m pixelpitch, 80% fill factor, and 250 mg/cm2 CsI:Tl x-ray con-verter. The bench incorporated computer-controlled transla-tion stages and a rotation stage �406XR and DynaservDM1060B, respectively, Parker Daedal, Harrison, PA�. Thegeometry was similar to systems for CBCT-guided radiationtherapy,11,38 with source-to-axis distance �SAD=93.5 cm�and source-to-detector distance �SDD=144.4 cm� yielding amagnification of �1.54 and a ��25.6�25.6�25.6 cm3�field of view. CBCT images were reconstructed from 320projections acquired in a circular orbit over 360° at 120 kVp�added filtration of 2.0 mm Al+1.1 mm Cu, chosen to re-duce beam-hardening effects and provide beam quality com-parable to that of clinical CT scanners�. The dose was variedvia five settings of mAs �0.4–1.25 mAs�. An antiscatter gridwas not employed.

The FDK reconstruction algorithm39 was used. Voxel sizewas isotroptic �0.25�0.25�0.25� mm3, determined by thedetector pixel pitch divided by the magnification, and thereconstruction filter consisted of a ramp and adjustableapodization window

Twin�f� = hwin + �1 − hwin�cos�2�fxinc� �1�

where f is the transverse frequency domain, xinc is the detec-tor pixel size, and the parameter hwin is an adjustableapodization parameter �hwin=0.5 for a smooth Hanning filter;hwin=1.0 for a sharp Ram-Lak36 filter�. The Hanning filterwas the nominal choice for all reconstructions, unless other-wise specified. No additional binning or filtering was used,and a scatter-correction algorithm was not employed.

II.A.2. Contrast-detail phantoms

Four custom-built 3D contrast-detail phantoms were used�The Phantom Laboratory, Greenwich, NY�. Each phantom

FIG. 2. Example ROIs illustrating �a� spheres of various size and contrast an�Ram-Lak� filters. ROIs in �a� are in the axial plane and were acquired at 1.2are of a 6.4 mm,105 HU sphere.

was a cylinder �150 mm diameter, 120 mm height� contain-

Medical Physics, Vol. 34, No. 11, November 2007

ing seven acrylic spheres �110 HU measured at 120 kVp ona clinical CT scanner—GE Lightspeed, General Electric Co.,Waukesha WI� of diameter 1.6–12.7 mm as in Fig. 1�b�. Thefour cylinders were formed of polyurethane mixed withvarying concentrations of antimony to provide four levels ofuniform background ranging from 64 to 275 HU. The result-ing contrast of the spheres �i.e., absolute difference in meanHU between the spheres �110 HU� and the uniform back-ground �64–275 HU�� for the four phantoms was 20, 46,105, and 165 HU �each ±5 HU, measured as the mean voxelvalue in a square ROI within the central slice of the largestsphere �to avoid partial volume effects�, on the clinical CTscanner�. These contrast values simulated a range of soft-tissue structures—e.g., contrast of gray to white matter inbrain ��20 HU�, prostate or breast to water ��46 HU�, fatto water ��105 HU�, and muscle to fat ��165 HU�. Uni-form backgrounds, especially within a small ROI containedin one tissue type, are a reasonable abstraction of variousclinical situations such as detection of a lesion within a solidorgan �e.g., a metastasis within the liver�. Such simple back-grounds are suitable for early investigations designed toprobe the fundamental limits of CBCT imposed by quantumnoise, reconstruction filter, etc.

The four phantoms �20–165 HU contrast� were imaged atfive levels of mAs �0.4–1.25 mAs�, for a total of 20 CBCTimages. Each image contained five diameters of sphere�1.6–12.7 mm� providing 100 combinations of contrast-diameter-dose �herein referred to as “cases”� for analysis ofimaging performance. Example images acquired on the im-aging bench are shown in Fig. 2, showing regions aboutspheres �a� of varying contrast and diameter and �b� at vary-ing dose in axial and sagittal planes reconstructed using Han-ning and Ram-Lak filters. As discussed in Sec. III A, CBCTimages acquired on the benchtop exhibit lower contrast com-pared to those obtained on a clinical CT scanner, primarily

� axial and sagittal images reconstructed using smooth �Hanning� and sharps per projection ��4–6 mGy total depending on the phantom.�. ROIs in �b�

d �b5 mA

due to scatter.

4462 Tward et al.: Soft-tissue detectability in cone-beam CT 4462

II.A.3. Imaging dose

The dose to isocenter was computed for the five mAslevels for each phantom. The dose to isocenter, for a watercylinder of equal diameter was computed as:7,40,41

Dwater � Nproje−�water�� mR

mAs��mAs��SF�f , �2a�

where the mR/mAs was measured to be 1.8 mR/mAs for thebench geometry and x-ray spectrum described above. Theradius of the phantom, �, was 75.25 mm, and a scatter factor�SF� of 4.5 and f-factor of 0.9 provided Dwater values consis-tent with doses measured at the center and periphery of a16 cm cylindrical dosimetry phantom.40 To correct for at-tenuation differences associated with each phantom, themean attenuation coefficient for each was estimated from theHU measured on the clinical CT scanner:

�phantom = �water�HUphantom

1000+ 1� , �2b�

and the dose to isocenter was corrected by the resulting at-tenuation:

Dphantom = Dwatere−�phantom�

e−�water�, �2c�

where �water=0.02 mm−1 is the linear attenuation coefficientfor water. Two assumptions associated with this approxima-tion were: �i� The SF and f-factor were constant for eachphantom, and �ii� the attenuation coefficient estimated fromHU on the clinical CT scanner was the same as on the ex-perimental benchtop—i.e., the x-ray spectra were equivalent.While such approximations could certainly be refined, theresulting dose values were consistent with other reports onCBCT dosimetry,40–42 and the correction in Eq. �2c� gavegood agreement between the measured and expected depen-dence of CBCT voxel noise on dose for each phantom. Theresulting dose to isocenter for the five settings of mAs lieswithin the range �1–7 mGy. These values are representa-tive of the range in CBCT-guided procedures—e.g., a dose of3 mGy and 10 mGy suggested for bony-tissue and soft-tissue visualization, respectively, in CBCT-guided head andneck surgery,40 significantly less than in diagnostic CT �Refs.43 and 44� and sufficiently low for repeat intraoperative �orinterfraction� scanning. Note that dose describes the dose towater at isocenter and accounts for attenuation of the back-ground, which is different for each phantom.

II.A.4. SDNR analysis

Prior to the 2AFC tests described below, the contrast andnoise in CBCT reconstructions were characterized: �i� Toensure that the image statistics follow a Poissondistribution—e.g., noise inversely proportional to the squareroot of dose,23 �ii� to separate factors related to simple per-formance metrics �contrast and noise� from more compli-cated factors investigated in the 2AFC tests—e.g., the influ-

ence of NPS correlation and prior information, and �iii� to

Medical Physics, Vol. 34, No. 11, November 2007

examine the 2AFC results relative to common rules of thumbfor detectability—viz., the Rose criterion.45–47

The largest sphere �12.7 mm� was used to analyze thecontrast and noise. As shown in Fig. 3, a 27�27 voxel ROIwas identified inside the sphere, and eight smaller ROIs �9�9 voxels� were identified surrounding the sphere. The ab-solute contrast, C, was defined as the mean signal difference:

C = ROIsphere − ROIbckgrnd, �2d�

and the relative contrast, Crel, was defined as the mean signaldifference normalized by mean voxel value:

Crel =ROIsphere − ROIbckgrnd

�ROIsphere + ROIbckgrnd�/2. �2e�

The absolute voxel noise, �, was given by the average stan-dard deviation in voxel values in the sphere and backgroundROIs:

� = ��sphere + �bckgrnd�/2, �2f�

and the relative voxel noise, �rel, was given by the absolutevoxel noise normalized by the mean voxel value:

�rel =��sphere + �bckgrnd�/2

�ROIsphere + ROIbckgrnd�/2. �2g�

The contrast-to-noise ratio �CNR�, or signal-difference-to-noise ratio �SDNR�, was therefore:

SDNR =C

�=

Crel

�rel=

ROIsphere − ROIbckgrnd

��sphere + �bckgrnd�/2. �2h�

Values for contrast and noise were calculated from ten con-secutive slices spanning 2.5 mm through the center of thesphere, with average value and error determined by the meanand standard deviation, respectively, over the nine ROIs and

FIG. 3. Example �a� axial and �b� sagittal ROIs within CBCT images of thelargest sphere �12.7 mm diameter�, with black squares overlain to showregions within the object and background for calculation of contrast andnoise. These ROIs also illustrate the difference in noise correlation in axialvs sagittal planes - bandpass vs asymmetric bandpass/lowpass, respectively.

ten slices.

4463 Tward et al.: Soft-tissue detectability in cone-beam CT 4463

II.B. Two-alternative forced choice „2AFC… tests

II.B.1. Experimental conditions and parameters

Thirteen tests were conducted in random order �see TableI� from which soft-tissue detectability could be evaluated forthe three specific studies detailed in Sec. II B 2. For eachcase, a pair of ROIs was presented—one containing a sphere�the “stimulus”� and one containing “noise-only”—and theobserver was asked to select �by mouse click� which of thetwo more likely contained the stimulus. The number of re-peats �3–25, as in Table I�, defined as the number of timesthe same case �from the same CBCT image� was presented toeach observer in a single test, was determined from pilotstudies that estimated response variability �fewer repeats re-quired for larger stimuli and/or higher dose�. Ideally, repeatswould be presented from different image acquisitions so thatdistinct realizations of quantum noise would average overcase sampling variability; however, practical limitations�time constraints and limited data storage� made this unfea-sible. Multiple repeats for each observer served to averageresponses over reader inconsistency. A total of 3300 ROIpairs were presented in the 13 tests. Training sets involved anadditional 30 ROI pairs presented prior to each test. Viewingtime was unrestricted, and short breaks were allowed as de-sired between tests �but not during�. An average of �2.3 hwas required to complete all tests, with an average of �2.5 sper decision, ranging from �1 s for presumably conspicuouscases to �5 s for more challenging detection tasks.

a. Image preprocessing. Prior to the tests, the location ofthe “stimulus” and “noise-only” ROIs was identified inCBCT reconstructions based on knowledge of phantom ge-ometry. A �125�125 voxel� ROI surrounding each location

TABLE I. Summary of stimuli and number of ROI pairs presented in 2AFC texample, the final row indicates 30 ROI pairs for 1.25 mAs, corresponding t�2 contrasts �5 diameters =30 ROI pairs.

ConditionTestNo.

Contrast�HU�

Diameter�mm�

Stimulus KnownExactly�SKE�

1 20 6.42 463 1054 1655 20 3.26 467 1058 165

Contrast Known,Size Unknown

�CKSU�

9 20 1.6, 3.2, 6.4, 9.5, 1

10 46 1.6, 3.2, 6.4, 9.5, 1

Contrast UnknownSize Known

�CUSK�

11 20, 46 3.2105,165

12 20, 46 6.4105,165

Stimulus Unknown�SUK�

13 20, 46 1.6, 3.2, 6.4, 9.5, 1105,165 1.6, 3.2, 6.4, 9.5, 1

was cropped to give stimulus and noise-only images. Care

Medical Physics, Vol. 34, No. 11, November 2007

was taken to ensure that the stimulus and noise-only ROIsexhibited the same noise characteristics and did not exhibitrecognizable features �e.g., ring artifacts� aside from thestimulus and quantum noise, respectively. Noise-only ROIswere generally identified at the same location as the stimulusin a nearby slice. This ensured that an observer could notbase his decisions on the orientation of ringlike noise relativeto the square ROI. For each case, one stimulus ROI waspaired with one noise-only ROI, and the same pair was pre-sented to each observer.

b. Observers and imaging task. Seven observers �physi-cists� were used. Previous studies48 suggest that for 2AFCtests involving simple phantom images �e.g., detection of asphere in a uniform background�, such observers performequivalently to expert radiologists. Identical instructionswere given to each observer.

c. Display and viewing conditions. ROI pairs were dis-played side by side on the same monitor, and surroundingregions of the screen were masked by black posterboard.Observers could not adjust the level, window, or magnifica-tion of the ROIs. The grayscale level was equal to the meanvoxel value within the ROI, and the grayscale window was±5 times the standard deviation in the stimulus image. Im-ages were displayed with positive contrast �i.e., bright lesion,dark background�. An LCD monitor �1280�1024 pixels; 96pixels/inch; 32 bits; Dell, Inc.� was sufficient to display im-ages without loss of resolution or gray-level discrimination,given the small format �125�125 voxels� and low grayscalerange of the displayed ROIs. Display brightness and contrastwere set to factory defaults. The room was darkened duringeach test, and a constant viewing distance of �50 cm was

The number of ROI pairs denotes the total for all cases in a given row. Fore repeats for each of the ten contrast-diameter combinations—i.e., 3 repeats

Number of ROI Pairs

0.4mAs

0.5mAs

0.63mAs

0.8mAs

1.25mAs Total

25 20 15 10 5 7525 20 15 10 5 7525 20 15 10 5 7525 20 15 10 5 7525 20 15 10 5 7525 20 15 10 5 7525 20 15 10 5 7525 20 15 10 5 75

125 100 75 50 25 375

125 100 75 50 25 375

100 80 60 40 20 45050 40 30 20 10

100 80 60 40 20 45050 40 30 20 10

200 160 120 80 40 1050150 120 90 60 30

ests.o thre

2.7

2.7

2.72.7

suggested but not strictly enforced.

4464 Tward et al.: Soft-tissue detectability in cone-beam CT 4464

d. Analysis of performance. In each 2AFC test, the pro-portion of correct responses �Pcorr� was analyzed as a figureof merit for soft-tissue detectability and evaluated as a func-tion of dose for spheres of varying size and contrast. The useof Pcorr as a figure of merit relates to the ROC curve—viz.,Pcorr ideally equal to the area under the ROC curve.37,49,50

Thus for a 2AFC test, Pcorr=0.5 implies pure guessing, andPcorr=1.0 implies perfect response. More specifically, Pcorr isthe probability that the decision variable produced by a “sig-nal � noise” ROI is greater than that produced by a noise-only ROI. Note that Pcorr was used in favor of detectabilityindex, d� �a signal to noise ratio defined for decision vari-ables�, as a figure of merit due to its direct relationship with2AFC tests, and because the calculation of d� requires as-sumptions as to the distribution of the decision variable.While d� has theoretical advantages, including linearity withdose, reporting in terms of d� is inapplicable to much of thedata �conspicuous cases� due to its divergence as Pcorr ap-proaches 1 �d�→ � �.

Because measured values of Pcorr are related to a binomialrandom variable whose variance is determined uniquely byits mean and sample size, standard deviations are a poorindication of uncertainty. Accordingly, 95% confidence inter-vals were calculated using bootstrapping with 10000 samplesdrawn from the combined responses of all observers. Thispresents meaningful error bars that reflect the asymmetry ofthe binomial distribution.

e. OPTEx - A software toolkit for observer performancetests. To streamline the various preprocessing, display,randomization, and analysis aspects of the 2AFC tests de-scribed above, a MATLAB program �The Mathworks, NatickMA� was developed,48 referred to as the Observer Perfor-mance Test Engine �OPTEx�. The software has four primaryfunctions: �i� It provides a simple interface to the image da-tabase by means of an Excel input parameter file listing thedirectories, filenames, ROI locations, ROI sizes, and associ-ated experimental parameters �e.g., dose, kVp, voxel size,etc.� for each case in a test; �ii� it randomizes the left-rightdisplay and reading order of ROI pairs; �iii� for each case, itrecords truth, observer response �mouse click�, and responsetime; and �iv� it analyzes mean and intraobserver standarddeviation in Pcorr �as well as the mean and standard deviationin observer response time� as a function of any experimentalparameter contained in the input parameter file.

The Excel input parameter file is generated with the as-sistance of an OPTEx preprocessing utility that provides agraphical interface for examining the 3D image data; select-ing the �x ,y ,z� coordinates for each stimulus and noise-onlyROI to be used in the test, and writing the coordinates to theinput file. In addition to randomizing left-right display andreading order of ROI pairs, the option exists for each ROI tobe randomly flipped horizontally and/or vertically �to reducememorization bias� and shifted �a random in-plane displace-ment, so that the stimulus does not necessarily appear in theexact center of the ROI�. Such additional randomization wasfound to reduce the tendency for observers simply to select

whichever image exhibited the brighter voxel value at the

Medical Physics, Vol. 34, No. 11, November 2007

exact center. Randomly shifting, rotating, and flipping theROIs added a slight search and localization aspect to thetask, increasing reader uncertainty and variability, which wasfound to make better use of the finite number of images,reduce observer fatigue and boredom, and provide a greaterproportion of useful data �i.e., Pcorr�1.0�.

II.B.2. Effects of dose, plane of visualization, andprior knowledge

a. Study #1: Low-dose detectability limits. Results fromall 13 tests in Table I were pooled to analyze the overallcontrast-detail-dose performance—i.e., evaluation of Pcorr asa function of dose for each size and contrast of sphere. Theresulting Pcorr versus dose data were fit to a two-parametercurve, Pcorr=a�1−e−bDose�. Fits were intended only as guidesfor the eyes within the range of doses examined, and do notrepresent a physical model. Fits of this form were chosen infavor of a sigmoid that approaches Pcorr=0.5 for zero dose,because the lack of data near the “toe” led to poor fits. Thedose required to achieve a given value of Pcorr �e.g., D70,defined as the dose required to achieve Pcorr=70%� wasevaluated from the curve fits and plotted versus the contrastand diameter of the sphere to quantify the low-dose soft-tissue detectability limits.

b. Study #2: Axial versus sagittal plane of visualization.Previous investigation20–22 of the 3D NPS of CBCT imagessuggests an asymmetric noise characteristic that imparts adistinct textural difference, as illustrated in Fig. 3. To quan-tify the influence of such asymmetry on the low-dose soft-tissue detectability limits, 2AFC tests were conducted inwhich the stimulus and noise-only ROIs were taken fromaxial or sagittal planes. All images were taken near the cen-tral slice �stimuli extending above and below the centralplane by, at most, the radius of the largest sphere, �6.4 mm�to avoid complications due to detectability changes as afunction of distance from the central slice �e.g., cone-beamartifact�. In addition, spheres presented to the observers wereobtained from five consecutive slices, reducing the potentialfor single-sample bias in a manner that could not be achievedin studies involving smaller spheres. The influence of thereconstruction filter was quantified by examining twoextremes—a smooth Hanning filter �hwin=0.5� and a sharpRam-Lak filter �hwin=1�. Examples of axial and sagittalROIs for the two choices of filter are illustrated in Fig. 2�b�.Each test consisted of 400 ROI pairs, corresponding to twoplanes of visualization, two choices of reconstruction filter,five dose levels, and 20 repeats, with all other aspects of thetest as described above.

A second effect can potentially degrade detectability insagittal planes—viz., the cone-beam artifact associated withviolation of Tuy’s condition for a circular orbit.51,52 The ar-tifact results in the misappropriation of signal power belong-ing to spatial frequencies in a range �constituting a “cone”�about the longitudinal axis in the 3D frequency domain, withextent of the cone determined by the system geometry.53,54

To minimize this effect �so that observed differences in de-

tectability may be attributed to NPS asymmetry rather than

4465 Tward et al.: Soft-tissue detectability in cone-beam CT 4465

cone-beam artifact� we note the following: �i� Stimuli werelocated in the central plane of 3D reconstruction, where thecone-beam artifact is negligible; �ii� stimuli were spherical,thus presenting a rich spectrum of signal power largely un-affected by the cone-beam artifact55,56 �as opposed, e.g., to aflat disk,57 which suffers the artifact completely�; �iii� thecontrast and SDNR �as shown in Sec. III A� were equivalentin axial and sagittal planes, demonstrating that there is noloss in contrast �signal power misappropriation� and no dif-ference in voxel noise; and �iv� qualitative visual inspectionreveals no noticeable artifact in the sagittal images acquiredunder these conditions �e.g., Fig. 3�b��. Therefore, differ-ences in Pcorr measured between axial and sagittal planes areattributed to 3D NPS asymmetry, and not cone-beam artifactor differences in contrast or SDNR.

c. Study 3: The effect of prior knowledge. As shown inTable I, tests varied in terms of the instructions provided tothe observer. In Tests 1–8, observers were informed exactlyof the stimulus to be identified �and were shown sampleimages of such in training prior to each test�—referred tobelow as SKE. In Tests 9–10, observers were told the con-trast of the stimulus �and shown example images of such intraining� but not the size–referred to as “Contrast-Known/Size-Unknown �CKSU�. Similarly in Tests 11 and 12, ob-servers were told the size of the stimulus but not thecontrast—referred to as “Contrast-Unknown/Size-Known�CUSK�”—and in Test 13, observers were told neither thecontrast nor size of the stimulus—referred to as SUK.

While prior information has been shown by many authorsto affect observer performance in general, differences probedherein were of interest with respect to various applications ofCBCT. For example, in CBCT-guided procedures �e.g., ra-diotherapy and surgery�, the observer typically has fullknowledge of the object being imaged �from preoperativeCT and MRI�, and the task is primarily to localize the targetfor precise delivery of a therapeutic agent—analogous toSKE. In diagnostic imaging, on the other hand, the observermay have little knowledge of the object to be detected �asidefrom clinical knowledge—e.g., that a certain lesion typicallyexhibits a certain shape or contrast�, and the task is to deter-mine what �if any� abnormality exists—analogous to SUK.For otherwise identical cases, it was therefore possible toevaluate differences in Pcorr versus dose resulting solely fromthe degree of prior knowledge via 2AFC tests conductedunder the SKE/CKSU/CUSK/SUK conditions. Future workwill involve LROC and ROC tests as more clinically realisticprobes of these distinct guidance and diagnosis tasks.

III. RESULTS

III.A. Signal-difference-to-noise ratio

The contrast, noise, and SDNR are plotted as a function ofdose in Fig. 4. Figure 4�a� confirms that contrast is indepen-dent of dose as expected. Moreover, the contrast of a givensphere appears to be the same between axial and sagittalplanes within experimental error. Ideally, one would expectan approximate eight-fold increase in contrast from the

20 HU phantom to the 165 HU phantom; however, the dif-

Medical Physics, Vol. 34, No. 11, November 2007

FIG. 4. Plots of �a� contrast, �b� noise, and �c� SDNR as a function of dose.Measurements correspond to the 12.7 mm sphere, with �a� linear, �b� inversesquare root, and �c� square root curve fits in each case. Analysis in axial andsagittal planes, as illustrated in Fig. 2, is represented by solid and opensymbols, respectively. The measurements suggest that contrast, noise, andSDNR are the same between planes of visualization �within experimentalerror�.

4466 Tward et al.: Soft-tissue detectability in cone-beam CT 4466

ference is approximately three fold �i.e., the measured rela-tive contrast for the 165 HU phantom was �3.2 timesgreater than the 20 HU phantom�. Contrast reduction is at-tributable to increased x-ray scatter relative to conditions onthe clinical scanner. Figure 4�b� confirms that voxel noisereduces as the inverse square root of dose, and the voxelnoise measured in axial and sagittal planes appears to be thesame within experimental error. The SDNR is plotted as afunction of dose in Fig. 4�c�. In each case, SDNR increasesas the square root of dose, and there is no appreciable differ-ence between axial and sagittal planes. Note that the SDNRfor all cases is less than three although many images couldbe correctly identified 100% of the time �Pcorr=1.0�. As dis-cussed below, this raises interesting consideration of theRose criterion46,47,58 �which suggests a threshold of �3–5for reliable detection� with respect to spatial frequencies ofinterest in the imaging task compared to the spatial-frequency characteristics of the background and NPS.

III.B. 2AFC tests

Figures 5–8 summarize the results of the 2AFC tests. Onecase �out of 100� appeared to be an outlier in the data andwas excluded from the analysis—viz., the case involving the�6.4 mm,46 HU� sphere imaged at �2 mGy was found ret-rospectively to exhibit a slight ring artifact in the noise-onlyimage that appeared similar to the stimulus, causing observ-

FIG. 5. Performance �Pcorr� measured as a function of dose. Each graph con20 HU, �b� 46 HU, and �c� 105 HU.

FIG. 6. Performance �Pcorr� measured �a� as a function of dose at various lev

levels �2.0–6.4 mGy�.

Medical Physics, Vol. 34, No. 11, November 2007

ers to mistakenly select the noise-only image. Also, Pcorr forthe 1.6 mm diameter spheres was typically in the range�0.5–0.6 �near guessing�, exhibited high variability, andwas excluded for purposes of clarity. Such low performancewas partly real �owing to physical performance challenges ofimaging the small stimulus� and partly artifactual �visibilitydegraded by slight ring artifacts which are most severe nearthe center of 3D reconstruction, where the 1.6 mm spherewas located, as shown in Fig. 1�b��.

III.B.1. Study I: Contrast-detail-dose detectabilitylimits

Figures 5 and 6 show Pcorr as a function of dose forstimuli of various size and contrast. Results in Fig. 5 corre-spond to spheres of a given contrast �20, 46, and 105 HU inFigs. 5�a�–5�c�, respectively� with separate curves plotted foreach diameter. Figure 6 presents the same data, with separatecurves plotted for �a� various contrasts and �b� various doselevels. The general trend throughout is as expected: Pcorr

increases with dose, contrast, and diameter. As shown in Fig.5�a� for the lowest-contrast sphere �20 HU�, Pcorr increasesfrom �0.5–0.6 �pure guessing� to �0.9–1.0 �perfect re-sponse� across the range of diameters 3.2–12.7 mm; there-fore, for this fairly challenging soft-tissue structure, stimulibelow �6 mm in size are poorly visualized, while stimuli9 mm are well seen, particularly at dose �3 mGy or

results for four sizes of sphere �3.2–12.7 mm diameter� and contrast of �a�

f contrast �20–165 HU� and �b� as a function of object size at various dose

tains

els o

4467 Tward et al.: Soft-tissue detectability in cone-beam CT 4467

greater. For a higher-contrast sphere �e.g., 105 HU as in Fig.5�c��, detectability of the 3.2 mm stimulus depended mark-edly on dose, whereas stimuli 6 mm diameter were wellseen at all dose levels investigated.

Similarly in Fig. 6�a�, the 6.4 mm sphere was well seen atall dose levels for higher-contrast stimuli �105 and 165 HU�.The detectability of more subtle stimuli �20 and 46 HU�,however, depended more strongly on dose, illustrating theimportance of knowledgeable, task-specific selection of im-aging techniques. For example �Fig. 6�a��, the detectabilityof the �6.4 mm,46 HU� sphere ranged from poor to con-spicuous over the range from �1 to �5 mGy.

III.B.2. Study #2: Axial versus sagittal plane ofvisualization

Figure 7 shows Pcorr versus dose for axial and sagittalplanes of visualization, for both “smooth” �Hanning� and“sharp” �Ram-Lak� reconstruction filters. As shown in Fig.7�a� �6.4 mm, 105 HU sphere�, there is a slight improvementin the axial plane compared to the sagittal. The difference isgreater for the sharp filter, as shown in Fig. 7�b�, owing togreater asymmetry in the 3D NPS. Cases involving moreconspicuous stimuli �Pcorr�1� and more challenging stimuli�Pcorr�0.5–0.6� exhibited a smaller difference in detectabil-

FIG. 7. Performance �Pcorr� measured in axial versus sagittal planes for �a� adetection of a �6.4 mm,105 HU� sphere is different between these planes offilter. As shown in �b�, the difference in detectability between axial and sag

FIG. 8. Performance �Pcorr� measured under various levels of prior knowled20 HU sphere� was essentially undetectable under SUK conditions, with a sa 6.4 mm, 20 HU sphere� exhibited a greater difference between SUK and S

there was no appreciable difference between levels of prior knowledge.

Medical Physics, Vol. 34, No. 11, November 2007

ity between axial and sagittal planes—appearing obvious orinvisible, respectively, in either domain regardless of theplane of visualization or reconstruction filter. Intermediatestimuli, however, were subject to the effect—particularly inassociation with sharp filters that increase the degree of NPSasymmetry.

The differences in detectability illustrated in Fig. 7 areattributed to NPS asymmetry and not SDNR or cone-beamartifact for reasons previously outlined. First, the cone-beamartifact is believed to be negligible under the conditions con-sidered. Second, as shown in Fig. 4, differences in contrast,noise, and SDNR are negligible. Third, the effect is amplifiedfor sharper reconstruction filters. The results are consistentwith previous investigations20–22 of the 3D NPS and detect-ability index34 that show: �i� Superior low-frequency NEQ inaxial planes;21 �ii� improved detectability index in axialplanes for low-frequency detection tasks;21 �iii� higher NEQfor systems with a low-pass presampling transfer character-istic in the longitudinal direction �viz., indirect versus direct-detection FPDS�;20 �iv� improved NEQ for such systems us-ing a smoother filter;20 and �v� overall improvement in theminimum detectable sphere size in axial versus sagittalplanes.21 Such studies involved analysis of the ideal ob-server, with results attributable solely to signal and noise

oth Hanning filter and �b� a sharp Ram-Lak filter. Results demonstrate thatlization, superior in the axial domain, and reduced for the sharp �Ram-Lak�lanes is increased for the sharper filter.

vided to the observer �See Table I�. �a� The most subtle stimulus �3.2 mm,boost under SKE conditions. �b� Stimuli of intermediate detectability �e.g.,nditions. �c� For more conspicuous stimuli �e.g., a 6.4 mm, 46 HU sphere�,

smovisuaittal p

ge promallKE co

4468 Tward et al.: Soft-tissue detectability in cone-beam CT 4468

transfer characteristics of the imaging system �rather thanmore complicated psychophysical effects�. The effects dem-onstrated in Fig. 7 may be accentuated beyond that duesolely to the 3D NEQ, with possible effects resulting fromdegradation in real observer performance under conditionswhere the background noise is asymmetric �combinedmidpass/low-pass characteristic in x /z directions of sagittalplanes� compared to cases where the background noise issymmetric �midpass characteristic in both x and y directionsin axial planes�—i.e., the asymmetric background noise tex-ture �as in Figs. 2�b� and 3� may be poorly matched to theeye response of a real observer.57

III.B.3. Study #3: The effect of prior knowledge

Figure 8 shows Pcorr as a function of dose under variouslevels of prior knowledge provided to the observer regarding�a� subtle, �b� intermediate, and �c� more conspicuousstimuli. For the most subtle stimuli �20 HU, 3.2 mm sphereas in Fig. 8�a��, performance under SKE conditions was el-evated at all dose levels compared to SUK conditions, frompure-guessing to Pcorr�0.6. For intermediate stimuli�20 HU, 6.4 mm sphere as in Fig. 8�b��, the improvement inperformance appears greater �e.g., Pcorr�0.85 for SKE, com-pared to �0.65 for SUK at the highest dose�. For more con-spicuous stimuli �46 HU, 6.4 mm sphere as in Fig. 8�c��,there was no appreciable difference in performance amongvarious levels of prior knowledge. Results for intermediatelevels of prior knowledge �CKSU and CUSK� were betweenthe SKE and SUK results, as might be expected, and wereexcluded from the graphs for clarity.

These results are interesting in at least two respects. First,the measured differences in Pcorr are not due to differences inimage quality, owing rather to psychophysical characteristicsfor known versus unknown signal detection tasks. Second,the results hold implications in relation to diagnostic versusimage-guidance applications where the degree of observerknowledge regarding the stimulus may vary in analogy toSKE and SUK. For challenging imaging tasks, knowledge ofthe stimulus can significantly elevate detectability above thelevel of pure-guessing �Fig. 8�a�� and indicate lower-dosetechniques at fixed detectability �e.g., a factor of 2 or more inD70 for SKE compared to SUK in Fig. 8�b��. The resultsillustrate a simple, intuitive fact that is well known in theperception literature that “if you know what you are lookingfor, it is easier find it” and, more importantly, quantifies thedosimetric implications in CBCT.

IV. DISCUSSION AND CONCLUSIONS

Concurrent with the development of new imaging tech-nologies over the last 20–30 years have come important ad-vances in the evaluation of imaging performance, includingstandardization of performance measurement, practical met-rics and physical models, and greater use and variations ofROC analysis. Still, the deceptively simple question pre-sented to the imaging scientist, Can I see it? �or more spe-cifically, At what dose can I see it? or from the standpoint of

statistical decision theory, Does the image more likely repre-

Medical Physics, Vol. 34, No. 11, November 2007

sent A or B?� is often elusive and requires quantitation of atleast three aspects of the imaging chain: �i� Precise definitionof the imaging task; �ii� characterization of the physical per-formance of the imaging system; and �iii� an understandingof the observer response. Particularly in the development ofnew technologies, an understanding of these factors is key tooptimization and knowledgeable clinical deployment. ForCBCT, characterization of physical performance metrics�e.g., SDNR and the 3D NPS or NEQ� together with evalu-ation of human observer performance �e.g., AFC tests� helpsto uncover the physical factors governing image quality andquantify their impact upon detectability.

The 2AFC tests quantify low-dose soft-tissue detectabilitytrends in CBCT and illustrate several points: �1� Confidentdetection �Pcorr70% � of a large low-contrast structure on auniform background �viz., a sphere of diameter much greaterthan the system correlation length� is achieved at contrast-to-noise ratio well below that suggested by the Rose criterion;�2� the detectability of a given object depends on the degreeand symmetry of noise correlation �e.g., axial versus sagittalplanes in CT�; and �3� such tests give a useful probe of therole of prior knowledge in such detection tasks �with impli-cations, e.g., in diagnostic versus image-guidance applica-tions�.

Figure 9 plots the dose required to achieve detection �D70,corresponding to Pcorr=70%� as a function of the diameterand contrast of the stimulus, as estimated from the curve fitsof Pcorr versus dose �as seen in Figs. 5 and 6�, with uncer-tainty calculated from error in fitting parameters. The overalltrends are intuitive �D70 declines with diameter and contrast�,but not obvious. The data suggest low-dose limits for task-specific CBCT techniques and reveal the dose at whichstimuli are confidently visualized with this new technology.

FIG. 9. Dose required to achieve a given level of detectability �viz., the doserequired to achieve Pcorr=70%� plotted as a function of size and contrast ofthe stimulus. The measurements suggest low-dose detectability limits forvarious size and contrast of soft-tissue structures. Curve fits are single-parameter curves of the form D70= �k /diameter�, intended as a guide for theeyes �not a physical model�.

Perhaps more importantly, the results help to establish to

4469 Tward et al.: Soft-tissue detectability in cone-beam CT 4469

what extent dose can be reduced for various tasks withoutcompromising detectability. Given the simplicity of theforced-choice detection task and the uniform background,the implied dose levels should be interpreted clinically aslower bounds—e.g., D70 for detection of a �6 mm,20 HU�sphere is �2 mGy. It is worth reiterating, however, that re-sults are based upon CBCT images acquired without an an-tiscatter grid and without a scatter correction algorithm; fur-ther improvement in SDNR �reduction in D70� can beexpected through improved management of x-rayscatter.59–61

Considering the detectability thresholds of Fig. 9 in rela-tion to the SDNR values of Fig. 4�c�, we see that detectabil-ity in these simple forced-choice detection tasks is achievedfor conditions corresponding to SDNR of �1, considerablybelow the Rose criterion �SDNR �3–5�. In such 2AFCtests, observers were clearly able to “detect” the stimulus�i.e., able to identify it in a binary choice� before it might beconsidered “visible” �in the colloquial sense�, with differ-ences between “detectability” and “visibility” owing to sev-eral factors. First, detection of a disk on a uniform back-ground corresponds to a low-frequency task function34 �e.g.,for a disk of diameter 3 mm, a first-order Bessel function,J1�f� / f , with first zero at �0.4 mm−1�. Therefore, the taskfunction is highest at the same frequencies that the �axial�NPS is lowest, resulting in improved detectability comparedto detection in white noise �as considered by Rose�. Simi-larly, detectability would be reduced �D70 increased� forcases of lumpy �or variable� background and higher-ordertasks �e.g., discrimination�, the former adding noise-power atlow frequencies and the latter involving a higher-frequencytask function �where the axial NPS is higher�. For the spe-cific case of anatomic backgrounds, detectability could bedecreased due to the 1/ f characteristic of anatomicalclutter,62–66 which has been described also in terms of a low-frequency reduction in the “generalized” DQE.67,68

The results suggest differences in detectability of a givenobject visualized in axial versus sagittal planes attributable toasymmetry in the 3D NPS, with the axial case exhibitingslightly higher detectability for the tasks considered. While itis important to recognize that different reconstruction algo-rithms affect the NPS in different ways, such asymmetry isnot unique to the FDK algorithm. The 3D NPS asymmetryexamined here is similar to that imparted by z-interpolationin helical CT. Conventional axial CT also involves 3D NPSasymmetry—midpass in the axial domain, compared to un-correlated noise in the longitudinal direction. The effect isconsistent with previous analysis of the 3D NEQ and detect-ability index,20,21 owing to the spatial-frequency content ofthe NPS relative to the imaging task: The axial NPS peaks atmidfrequencies, whereas the longitudinal �z� component ofthe sagittal/coronal NPS is purely low-pass; therefore, de-tectability of a disk in a uniform background is improved forthe former case. That is not to say that the axial domain isgenerally superior; on the contrary, it suggests that higher-order tasks involving middle and high frequencies �e.g., dis-

crimination, size estimation, etc.� may perform better in the

Medical Physics, Vol. 34, No. 11, November 2007

sagittal domain. Similarly, the low-pass characteristic of thesagittal NPS may provide superior performance for tasks in-volving lumpy and/or variable background. Given the grow-ing clinical utilization of “3D contouring” �e.g., in radio-therapy guidance� and multiplanar views �e.g., coronalreadings of MDCT� it is important to recognize that a givenobject can exhibit different detectability in different domainsof visualization owing to the asymmetric characteristic of the3D NPS.

The 2AFC tests provided insight into the role of priorknowledge on detectability, suggesting a significant differ-ence between SKE and SUK conditions for objects near thedetectability limit. While the results pertain to a simple de-tection task, the results hold implication for a variety of di-agnostic and image-guided applications seeking dosereduction—for example: �i� Follow-up screening of lungnodules via low-dose CT �for which the task is typically sizeestimation, and prior knowledge �CKSU� is gained from theoriginal screening exam�; and �ii� image-guided surgery orradiation therapy via CBCT �for which the task is typicallylocalization, and prior knowledge �SKE� is available fromdiagnostic and planning data�. The results suggest the poten-tial for dose reduction in situations where the observer hasprior knowledge of the stimulus compared to that in whichhe or she does not. For example, D70 for a �6.4 mm, 20 HU�sphere was �2.5 mGy under SKE conditions, compared to�5 mGy or greater for SUK conditions.

While the results pertain to fairly simple and idealizedtests, the approach of linking physical performance metrics�SDNR and NPS� to real observer performance �Pcorr� is use-ful in identifying low-dose techniques and guiding clinicaldeployment in a manner that explicitly considers the imagingtask. The observed low-dose soft-tissue detectability limitsare consistent with previous investigation in CBCT-guidedhead and neck surgery,40 and suggest the potential for furtherdose reduction in CBCT-guided radiation therapy.41 The re-sults are generally consistent with those of Boone et al.42 fordiagnostic CBCT imaging of the breast at �4–9 mGy, rec-ognizing differences in imaging geometry, beam energy, ob-ject size, etc. Implications for CBCT of the head and neck indental-maxillofacial applications are clear, with low-doselimits for bone and soft-tissue visualization of �5 mGy—abit higher ��8 mGy� for smaller and more subtle featuresand a bit lower ��2–3 mGy� for larger and more conspicu-ous features. The influence of 3D NPS correlation on detect-ability among various planes of visualization should be con-sidered in each of these applications, recognizing thepotential for different performance in axial versus sagittal/coronal planes, depending upon the imaging task. Futurework includes evaluation of imaging performance in phan-toms appropriate to larger body sites �e.g., chest, abdomen,and pelvis�, for which the influence of x-ray scatter, beamhardening, object truncation, and similar artifacts are consid-erably increased.

ACKNOWLEDGMENTS

The authors extend their gratitude to S. Ansell and G.

Wilson for assistance with software and instrumentation on

4470 Tward et al.: Soft-tissue detectability in cone-beam CT 4470

the experimental bench. T. Rosewall and V. Chhin �PrincessMargaret Hospital� assisted with measurements on the clini-cal CT scanner. Dr. K. Brown �Elekta Oncology Systems,Atlanta, GA� and Dr. M. Gauer �PerkinElmer Optoelectron-ics, Palo Alto, CA� provided the flat-panel detector and as-sociated technical information. The imaging phantoms wereconstructed in collaboration with J. Levy �The PhantomLaboratory, Greenwich, NY�. Useful discussions with Dr. J.M. Boone �UC Davis�, Dr. C. K. Abbey �UC Santa Barbara�,and Dr. A. E. Burgess �Harvard University, retired� are grate-fully acknowledged. This work was supported by the Na-tional Institutes of Health �Grant No. R01-CA112163-01�,the Radiology Research Fund at Toronto General Hospital,and a University of Toronto New Staff Award �No.72022001�. D.J.T was the recipient of a Harold Johns Stu-dentship from Cancer Care Ontario.

a�Author to whom correspondence should be addressed. Electronic mail:jsiewerd@uhnres.utoronto.ca

1W. Ross, D. D. Cody, and J. D. Hazle, “Design and performance charac-teristics of a digital flat-panel computed tomography system,” Med. Phys.33�6�, 1888–1901 �2006�.

2B. Chen and R. Ning, “Cone-beam volume CT breast imaging: Feasibilitystudy,” Med. Phys. 29�5�, 755–770 �2002�.

3J. M. Boone, N. Shah, and T. R. Nelson, “A comprehensive analysis ofDgN�CT� coefficients for pendant-geometry cone-beam breast computedtomography,” Med. Phys. 31�2�, 226–235 �2004�.

4R. Fahrig, M. Moreau, and D. W. Holdsworth, “Three-dimensional com-puted tomographic reconstruction using a C-arm mounted XRII: correc-tion of image intensifier distortion,” Med. Phys. 24�7�, 1097–1106 �1997�.

5J. H. Siewerdsen, D. A. Jaffray, G. K. Edmundson, W. P. Sanders, J. W.Wong, and A. Martinez, “Flat panel cone-beam CT: A novel imagingtechnology for image-guided procedures,” Proc. SPlE Visualization, Dis-play, and Image-Guided Procedures 4319, 435–444 �2001�.

6M. A. Rafferty, J. H. Siewerdsen, Y. Chan, D. J. Moseley, M. J. Daly, D.A. Jaffray, and J. C. Irish, “Investigation of C-arm cone-beam CT-guidedsurgery of the frontal recess,” Laryngoscope 115�12�, 2138–2143 �2005�.

7J. H. Siewerdsen, D. J. Moseley, S. Burch, S. K. Bisland, A. Bogaards, B.C. Wilson, and D. A. Jaffray, “Volume CT with a flat-panel detector on amobile isocentric c-arm: pre-clinical investigation in guidance of mini-mally invasive surgery,” Med. Phys. 32�1�, 241–254 �2005�.

8J. H. Siewerdsen, Y. Chan, D. J. Moseley, S. M. Kim, D. A. Jaffray, andJ. A. Irish, “A mobile, isocentric c-arm for cone-beam CT-guided headand neck tumor surgery,” Proc. SPIE Visualization, Display, and Image-Guided Procedures, 5744, 789–797 �2005�.

9M. A. Rafferty, J. H. Siewerdsen, Y. Chan, M. J. Daly, D. J. Moseley, D.A. Jaffray, and J. C. Irish, “Intraoperative cone-beam CT for guidance oftemporal bone surgery,” Otolaryngol.-Head Neck Surg. 134�5�, 801–808�2006�.

10G. Lauritsch, J. Boese, L. Wigstrom, H. Kemeth, and R. Fahrig, “Towardscardiac C-arm computed tomography,” IEEE Trans. Med. Imaging 25�7�,922–934 �2006�.

11D. A. Jaffray, J. H. Siewerdsen, J. W. Wong, and A. A. Martinez, “Flat-panel cone-beam computed tomography for image-guided radiationtherapy,” Int. J. Radiat. Oncol., Biol., Phys. 53�5�, 1337–1349 �2002�.

12D. Letourneau, J. W. Wong, M. Oldham, M. Gulam, L. Watt, D. A.Jaffray, J. H. Siewerdsen, and A. A. Martinez, “Cone-beam-CT guidedradiation therapy: technical implementation,” Radiother. Oncol. 75�3�,279–286 �2005�.

13M. H. Smitsmans, J. de Bois, J. J. Sonke, A. Betgen, L. J. Zijp, D. A.Jaffray, J. V. Lebesque, and M. Van Herk, “Automatic prostate localiza-tion on cone-beam CT scans for high precision image-guided radio-therapy,” Int. J. Radiat. Oncol., Biol., Phys. 63�4�, 75–984 �2005�.

14J. J. Sonke, L. Zijp, P. Remeijer, and M. Van Herk, “Respiratory corre-lated cone beam CT,” Med. Phys. 32�4�, 1176–1186 �2005�.

15Y. Sun, Y. Yan, and X. Chen, “Analysis and evaluation of the MTF ofcone-beam CT,” Proc. SPIE 5918, 591815-1–591815-12 �2005�.

16

J. H. Siewerdsen and D. A. Jaffray, “Optimization of x-ray imaging ge-

Medical Physics, Vol. 34, No. 11, November 2007

ometry �with specific application to flat-panel cone-beam computed to-mography�,” Med. Phys. 27�8�, 1903–l914 �2000�.

17A. L. Kwan, J. M. Boone, K. Yang, and S. Y. Huang, “Evaluation of thespatial resolution characteristics of a cone-beam breast CT scanner,” Med.Phys. 34, 275–281 �2006�.

18M. M. Thornton and M. J. Flynn, “Measurement of the spatial resolutionof a clinical volumetric computed tomography scanner using a spherephantom,” Proc. SPIE Physics of Medical Imaging, 6142, 61421Z-1–61421Z-10 �2006�.

19J. H. Siewerdsen and D. A. Jaffray, “Cone-beam CT with a flat-panelimager: Noise considerations for fully 3D computed tomography,” Proc.SPIE Physics of Medical Imaging 3977, 408–416 �2000�.

20J. H. Siewerdsen and D. A. Jaffray, “Three-dimensional NEQ transfercharacteristics of volume CT using direct and indirect-detection flat-panelimagers,” Proc. SPIE Physics of Medical Imaging 29�11�, 2655–2671�2003�.

21J. H. Siewerdsen, D. J. Moseley, and D. A. Jaffray, “Incorporation of taskin 3D imaging performance evaluation: the impact of asymmetric NPS ondetectability,” Proc. SPIE Physics of Medical Imaging 5368, 89–97�2004�.

22J. H. Siewerdsen, I. A. Cunningham, and D. A. Jaffray, “A framework fornoise-power spectrum analysis of multidimensional images,” Med. Phys.29�11�, 2655–2671 �2002�.

23H. H. Barrett, S. K. Gordon, and R. S. Hershel, “Statistical limitations intransaxial tomography,” Comput. Biol. Med. 6�4�, 307–323 �1976�.

24K. M. Hanson, “Detectability in computed tomographic images,” Med.Phys. 6�5�, 441–451 �1979�.

25M. F. Kijewski and P. F. Judy, “The noise power spectrum of CT images,”Phys. Med. Biol. 32�5�, 565–575 �1987�.

26R. Van Metter and D. Foos, “Enhanced latitude for digital projectionradiography,” Proc. SPIE Image Display 3658, 468–483 �1999�.

27C. K. Abbey and H. H. Barrett, “Human- and model-observer perfor-mance in ramp-spectrum noise: effects of regularization and object vari-ability,” J. Opt. Soc. Am. A 18�3�, 473–488 �2001�.

28C. K. Abbey and M. P. Eckstein, “Derivation of a detectabillity index forcorrelated responses in multiple alternative forced-choice experiments,” J.Opt. Soc. Am. A 17�11�, 2101–2104 �2000�.

29A. E. Burgess, “Comparison of receiver operating characteristic andforced-choice observer performance measurement methods,” Med. Phys.22�5�, 643–655 �1995�.

30M. P. Eckstein, C. K. Abbey, and F. O. Bochud, “Visual signal detectionin structured backgrounds. IV. Figures of merit for model performance inmultiple-alternative forced-choice detection tasks with correlated re-sponses,” J. Opt. Soc. Am. A 17�2�, 206–217�2000�.

31C. E. Metz, “Basic principles of ROC analysis,” Semin Nucl. Med. 8�4�,283–298 �1978�.

32J. A. Swets, “ROC analysis applied to the evaluation of medical imagingtechniques,” Invest. Radiol. 14�2�, 109–121 �1979�.

33D. M. Green and J. A. Swets, Signal Detection Theory and Psychophys-ics, John Wiley & Sons, New York �1966�.

34ICRU Report 54. Medical Imaging - The Assessment of Image Quality,International Commission on Radiation Units and Measurements, Be-thesda, MD �1996�.

35D. F. Habets, B. E. Chapman, A. J. Fox, D. Hyde, and D. W. Holdsworth,“Two-alternative forced choice evaluation of 3-D CT angiograms,” Proc.SPlE Image Perception and Performance 4324, 195–203 �2001�.

36B. E. Chapman, A. R. Sanderson, C. Goodrich, A. L. Alexander, D. D.Blatter, and D. L. Parker, “Observer performance methodologies forevaluating blood vessel visibility in MR angiograms using accurate geo-metric registration to high resolution x-ray angiograms,” Magn. Reson.Med. 37�4�, 519–529 �1997�.

37A. E. Burgess, “On sampling statistics in observer performance studies,”Proc. SPlE Science and Engineering of Medical Imaging, 1137, 190–197�1989�.

38D. A. Jaffray and J. H. Siewerdsen, “Cone-beam computed tomographywith a flat-panel imager: initial performance characterization,” Med.Phys. 27�6�, 1311–1323 �2000�.

39L. A. Feldkamp, L. C. Davis, and J. W. Kress, “Practical cone-beamalgorithm,” J. Opt. Soc. Am. A 1�6�, 612–619 �1984�.

40M. J. Daly, J. H. Siewerdsen, D. J. Moseley, D. A. Jaffray, and J. C. Irish,“Intraoperative cone-beam CT for guidance of head and neck surgery:Assessment of dose and image quality using a C-arm prototype,” Med.

Phys. 33�10�, 3767–3780 �2006�.

4471 Tward et al.: Soft-tissue detectability in cone-beam CT 4471

41M. K. Islam, T. G. Purdie, B. D. Norrlinger, H. Alasti, D. J. Moseley, M.B. Sharpe, J. H. Siewerdsen, and D. A. Jaffray, “Patient dose from kilo-voltage cone beam computed tomography imaging in radiation therapy,”Med. Phys. 33�6�, 1573–1582 �2006�.

42J. M. Boone, A. L. Kwan, J. A. Seibert, N. Shah, K. K. Lindfors, and T.R. Nelson, “Technique factors and their relationship to radiation dose inpendant geometry breast CT,” Med. Phys. 32�12�, 3767–3776 �2005�.

43A. C. Maclennan and D. M. Hadley, “Radiation dose to the lens fromcomputed tomography scanning in a neuroradiology department,” Br. J.Radiol. 68�805�, l9–22 �1995�.

44W. Huda, K. A. Lieberman, J. Chang, and M. L. Roskopf, “Patient sizeand x-ray technique factors in head computed tomography examinations.I. Radiation doses,” Med. Phys. 31�3�, 588–594 �2004�.

45A. Rose, “Quantum and noise limitations of the visual process,” J. Opt.Soc. Am. 43�9�, 715–716 �1953�.

46A. Rose, “Quantum effects in human vision,” Adv. Biol. Med. Phys. 5,211–242 �1957�.

47A. E. Burgess, “The Rose model, revisited,” J. Opt. Soc. Am. A 16�3�,633–646 �1999�.

48J. H. Siewerdsen, N. S. Paul, R. Wong, D. A. Patsios, A. T. Rao, T.Chung, and D. A. Jaffray, “Quantitative assessment of nodule detectabil-ity in chest CT: Delving the low-dose limits,” Proceedings of the Radio-logical Society of North America �RSNA, 90th Scientific Assembly�, 399�2004�.

49C. E. Metz, “Some practical issues of experimental design and data analy-sis in radiological ROC studies,” Invest. Radiol. 24�3�, 234–245 �1989�.

50J. A. Hanley and B. J. McNeil, “The meaning and use of the area under areceiver operating characteristic �ROC� curve,” Radiology 143�1�, 29–36�1982�.

51H. K. Tuy, “An inversion formula for cone-beam reconstruction,” SIAMJ. Appl. Math. 43�3–4�, 546–552 �1983�.

52B. D. Smith, “Image reconstruction from cone-beam projections: Neces-sary and sufficient conditions and reconstruction methods,” IEEE Trans.Med. Imaging 4, 14–25 �1985�.

53P. Grangeat, “Mathematical Framework for cone-beam reconstruction viathe first derivative of the radon transform,” Lect. Notes Math. l497,66–97 �1991�.

54X. H. Yan and R. M. Leahy, “Derivation and analysis of a filtered back-projection algorithm for cone beam projection data,” IEEE Trans. Med.Imaging 10�3�, 462–472 �1991�.

55D. A. Jaffray, R. Clackdoyle, J. H. Siewerdsen, and F. Noo, “Dependence

of cone-beam CT artifacts on the frequency content of the imaged ob-

Medical Physics, Vol. 34, No. 11, November 2007

ject,” Med. Phys. 30�6�, 1407 �2003�.56S. Bartolac, F. Noo, R. Clackdoyle, D. J. Moseley, J. H. Siewerdsen, and

D. A. Jaffray, “A local Fourier description of artifacts in circular cone-beam computed tomography,” Med. Phys. 33�6�, 2287 �2006�.

57M. Defrise, D. W. Townsend, and R. Clack, “Three-dimensional imagereconstruction from complete projections,” Phys. Med. Biol. 34�5�, 573–587 �1989�.

58A. Rose, “Quantum and noise limitations of the visual process,” J. Opt.Soc. Am. 43�9�, 715–716 �1953�.

59G. Jarry, D. Moseley, D. Jaffray, and F. Verhaegen, “Monte Carlo inves-tigation of scatter contribution to kilovoltage cone-beam computed to-mography Images,” Med. Phys. 32�6�, 2092 �2005�.

60J. H. Siewerdsen, M. J. Daly, B. Bakhtiar, D. J. Moseley, S. Richard, H.Keller, and D. A. Jaffray, “A simple, direct method for x-ray scatterestimation and correction in digital radiography and cone-beam CT,”Med. Phys. 33�1�, 187–197 �2006�.

61L. Spies, M. Ebert, B. A. Groh, B. M. Hesse, and T. Bortfeld, “Correctionof scatter in megavoltage cone-beam CT,” Phys. Med. Biol. 46�3�, 821–833 �2001�.

62F. O. Bochud, C. K. Abbey, and M. P. Eckstein, “Visual signal detectionin structured backgrounds. III. Calculation of figures of merit for modelobservers in statistically nonstationary backgrounds,” J. Opt. Soc. Am. A17�2�, 193–205 �2000�.

63A. E. Burgess, F. L. Jacobson, and P. F. Judy, “Human observer detectionexperiments with mammograms and power-law noise,” Med. Phys. 28�4�,419–437 �2001�.

64J. J. Heine and R. P. Velthuizen, “Spectral analysis of full field digitalmammography data,” Med. Phys. 29, 647–661 �2001�.

65S. J. Glick, S. Vedantham, and A. Karellas, “Investigation of optimal KVpSettine for CT mammography using a flat-panel imager,” Proc. SPIEPhysics of Medical Imaging 4682, 392–402 �2002�.

66S. Richard, J. H. Siewerdsen, D. Jaffray, D. J. Moseley, and B. Bakhtiar,“Generalized DQE analysis of radiographic and dual-energy imaging us-ing flat-panel detectors,” Med. Phys. 32�5�, 1397–1413 �2005�.

67R. F. Wagner, K. J. Myers, M. J. Tapiovaara, D. G. Brown, and A. E.Burgess, “Maximum a posteriori detection and figures of merit for detec-tion under uncertainty,” Proc. SPIE Medical Imaging IV, 195–204 �1990�.

68H. H. Barrett, J. L. Denny, R. F. Wagner, and K. J. Myers, “Objectiveassessment of image quality. II. Fisher information, Fourier crosstalk, andfigures of merit for task performance,” J. Opt. Soc. Am. A 12�5�, 834–852

�1995�.