An Assessment of Repeatability

for Crown Measurements Taken

on Conifer Tree Species

James A. WestfallWilliam A. Bechtold

KaDonna C. Randolph

USDA Forest ServiceUSDA Forest ServiceForest Inventory and AnalysisForest Inventory and Analysis

FIA QA/QC Data CollectionFIA QA/QC Data Collection

• Hot Checks• Cold Checks• Blind Checks

• Independent Plot Remeasurement• Randomly Chosen Plots• Experienced Personnel• Target 3%

Conifer DataConifer DataFIA Tree Crown Indicator (Phase 3)

• Uncompacted Crown Ratio (nearest 1%)

• Light Exposure (6 categories)

• Crown Position (4 categories)

• Crown Vigor Class (saplings – 3 categories)

• Crown Density (nearest 5%)

• Dieback (nearest 5%)

• Foliage Transparency (nearest 5%)

AnalysisAnalysis• Data Matching (i.e., trees)

No one-to-one correspondence (independent remeasure)

Manually expensive (large # observations)

Automate most matches– 2-pass approach

– Weighted distance = f(dbh, horz. distance, azimuth)

– ‘Conservatism’ via a decision rule

MUST review unmatched trees and add legitimate matches into analysis data set

AnalysisAnalysisMQOs and Tolerances

• Tolerance• A range of acceptable variation

• Can be specific value or percentage

• Example: ± 0.1 in. for dbh

• Measurement Quality Objective (MQO)• The desired percentage of measurements that fall

within the tolerance range

• Example: 95% of the time

AnalysisAnalysisComputations

• Obtain differences between field and QA crews for matched observations

• Determine percentage of total observations where difference is within the tolerance range

• Compare with MQO to see if standard is met

• Optional: compute percentages across range of tolerance values

ResultsResults

Variable Tolerance MQO @1x @2x @3x @4x RecordsUncompacted CR ±10 % 90 81.9% 94.7% 98.1% 99.6% 1350

Crown Light Exposure ±1 class 85 85.2% 93.0% 96.0% 97.3% 1350Crown Position No Tolerance 85 77.3% 1350

Vigor Class No Tolerance 90 81.5% 259Crown Density ±10 % 90 68.6% 91.1% 98.6% 99.5% 1091Crown Dieback ±10 % 90 97.7% 99.5% 99.6% 99.8% 1091

Foliage Transparency ±10 % 90 95.5% 98.8% 99.5% 99.9% 1091

Percentage of data within tolerance

*

ResultsResultsRegion Variable Tolerance

% within tolerance

Mean difference Records

NRS Uncompacted CR ±10 % 81.8 0.4 395SRS Uncompacted CR ±10 % 75.0 2.1 176IW Uncompacted CR ±10 % 84.2 -0.3 727PNW Uncompacted CR ±10 % 73.1 0.1 52

NRS Crown Light Exposure ±1 class 85.1 0 395SRS Crown Light Exposure ±1 class 86.9 -0.1 176IW Crown Light Exposure ±1 class 84.2 -0.2 727PNW Crown Light Exposure ±1 class 94.2 0.1 52

NRS Crown Position No Tolerance 84.6 -0.1 395SRS Crown Position No Tolerance 93.2 0 176IW Crown Position No Tolerance 69.2 0.1 727PNW Crown Position No Tolerance 80.8 -0.1 52

NRS Vigor Class No Tolerance 85.7 0 189SRS Vigor Class No Tolerance 76.9 -0.1 26IW Vigor Class No Tolerance 65.9 0.2 44PNW Vigor Class No Tolerance . . _

ResultsResultsRegion Variable Tolerance

% within tolerance

Mean difference Records

NRS Crown Density ±10 % 64.6 -5.1 206SRS Crown Density ±10 % 82.0 1.1 150IW Crown Density ±10 % 67.3 0.4 683PNW Crown Density ±10 % 61.5 -5.8 52

NRS Crown Dieback ±10 % 98.1 0.4 206SRS Crown Dieback ±10 % 97.3 -0.6 150IW Crown Dieback ±10 % 97.7 0.3 683PNW Crown Dieback ±10 % 98.1 0.4 52

NRS Foliage Transparency ±10 % 95.6 0.7 206SRS Foliage Transparency ±10 % 88.7 -1.1 150IW Foliage Transparency ±10 % 96.6 -0.4 683PNW Foliage Transparency ±10 % 100.0 -2.6 52

Uncompacted CRUncompacted CR

QA crew Tolerance% within tolerance

Mean difference Records

5.0% ±10 % 100.0 0 110.0% ±10 % 50.0 -16.5 215.0% ±10 % 50.0 -24.3 420.0% ±10 % 60.0 -11.1 1025.0% ±10 % 76.9 -4.2 2630.0% ±10 % 71.9 -6.3 3235.0% ±10 % 94.4 1.2 5440.0% ±10 % 79.7 -1.6 7945.0% ±10 % 78.7 -1.6 8950.0% ±10 % 77.5 -2.9 8955.0% ±10 % 82.3 1.1 7960.0% ±10 % 70.8 2.9 6565.0% ±10 % 61.4 -1.9 7070.0% ±10 % 77.5 0.9 7175.0% ±10 % 64.3 1.2 7080.0% ±10 % 76.9 0.1 9185.0% ±10 % 80.3 2.6 7690.0% ±10 % 89.2 1.9 11195.0% ±10 % 90.6 2.7 117

100.0% ±10 % 97.7 1.4 214

Vigor ClassVigor Class

QA crew Tolerance MQO% within tolerance Records

1 No Tolerance 90 88.8 2152 No Tolerance 90 51.3 393 No Tolerance 90 0.0 5

Crown DensityCrown Density

QA crew Tolerance MQO% within tolerance

Mean difference Records

0.0% ±10 % 90 0.0 -50 15.0% ±10 % 90 66.7 -11.7 3

10.0% ±10 % 90 60.0 -10 515.0% ±10 % 90 44.4 -15.6 920.0% ±10 % 90 75.0 -8.8 2025.0% ±10 % 90 73.8 -8.3 4230.0% ±10 % 90 65.5 -5.7 8435.0% ±10 % 90 77.8 -2.7 17140.0% ±10 % 90 67.8 -3.7 19945.0% ±10 % 90 74.9 -1 17550.0% ±10 % 90 69.0 0 11655.0% ±10 % 90 71.3 1.8 8760.0% ±10 % 90 66.7 5.3 7565.0% ±10 % 90 51.1 7.4 4770.0% ±10 % 90 50.0 10.5 2875.0% ±10 % 90 40.0 15.7 1580.0% ±10 % 90 27.3 18.6 1185.0% ±10 % 90 0.0 23.3 3

Crown DiebackCrown Dieback

QA crew Tolerance MQO% within tolerance

Mean difference Records

0.0% ±10 % 90 99.2 -1.4 7135.0% ±10 % 90 98.2 1.7 27310.0% ±10 % 90 100.0 5.3 7615.0% ±10 % 90 69.2 7.7 1320.0% ±10 % 90 60.0 3 525.0% ±10 % 90 100.0 10 130.0% ±10 % 90 33.3 8.3 335.0% ±10 % 90 0.0 -15 1

**45.0% ±10 % 90 0.0 45 150.0% ±10 % 90 0.0 35 1

**65.0% ±10 % 90 0.0 55 170.0% ±10 % 90 100.0 5 1

**90.0% ±10 % 90 0.0 25 2

Foliage TransparencyFoliage Transparency

QA crew Tolerance MQO% within tolerance

Mean difference Records

0.0% ±10 % 90 25.0 -17.5 45.0% ±10 % 90 66.7 -10 310.0% ±10 % 90 98.6 -3.9 21415.0% ±10 % 90 97.6 -0.9 46520.0% ±10 % 90 96.6 1.7 26425.0% ±10 % 90 86.4 2.5 10330.0% ±10 % 90 84.4 1.9 3235.0% ±10 % 90 75.0 3.8 4

**50.0% ±10 % 90 0.0 35 1

**95.0% ±10 % 90 0.0 80 1

ConclusionConclusion

Crown Light Exposure, Crown Dieback, and Foliage Transparency measurements met the stated repeatability standard.

Uncompacted CR, Crown Position, Crown Vigor, and Crown Density measurements did not meet the repeatability standard.

ConclusionConclusion• With few exceptions, levels of repeatability

are similar across geographic regions.

• The poorest repeatability statistics were generally associated with relatively rarer crown characteristics.

• For some variables, improved training and/ or re-evaluation of the tolerance/MQO may be needed.

ConclusionConclusion

• Quality assurance data are important for:• Evaluating training effectiveness• Employee performance feedback• Evaluating measurement protocols• Identifying significant sources of error for

computed attributes, model projections, etc.

ConclusionConclusion• Further reading:

• Westfall, J.A., ed. 2009. FIA national assessment of data quality for forest health indicators. USDA For. Serv. Gen. Tech. Rep. NRS-53.

• Pollard, J.E., et al. 2006. FIA national data quality assessment report for 2000-2003. USDA For. Serv. Gen. Tech. Rep. RMRS-181.

Questions ??Questions ??