CSv2 Reliability Study Final Reportcancerstaging.org/sites/csqatest/software/Documents/CSv2 Reliabilit… · 2011 CSv2 Reliability Study Final Report Pg. 3 of 32 BACKGROUND The Collaborative

2011 CSv2 Reliability Study Final Report

Pg. 1 of 32

FINAL REPORT AND RECOMMENDATIONS CSv2 RELIABILITY STUDY

September 2012

Report Authors: CSv2 Field Study Team Editors: Jerri Linn Phillips, MA, CTR Patricia Murison, CHIM, RHIT


Pg. 2 of 32

TABLE OF CONTENTS Section

Page Number

Background 3 Study Objective 4 Introduction 4 Preliminary Steps 5 Study Participants 5 Data Items 8 Reconciliation 9 Major, Minor and Unknown to Known Errors 12 Analysis 16 Conclusion 29 Recommendations 29

Protocol, Appendices and Tables located on http://cancerstaging.org/cstage

Appendix A: Timeline Appendix B: Invitation to Participate Appendix C: Users Guide

Appendix D: Demographics Appendix E: Site Specific Factors by Site Appendix F: Functional Requirements Document Appendix G: Post Participant Questions Appendix H: Call for Cases Appendix I: Case Assignments Appendix J: Team Members Appendix K: Case Review Worksheets Appendix L: Team Responsibilities and Detailed Timeline Appendix M: Request for CEUs Appendix N: Data Analysis Plan Appendix O: Frequency Reports Appendix P: Error Definitions


Pg. 3 of 32

BACKGROUND The Collaborative Staging (CS) Task Force was formed in 1998 to address discrepancies in staging

guidelines among the three major cancer staging systems used in the United States. The project was sponsored by the American Joint Committee on Cancer (AJCC) in collaboration with the); American College of Surgeons Commission on Cancer (CoC); the Canadian Council of Cancer Registries (CCCR); the Centers for Disease Control and Prevention National Program of Cancer Registries (CDC/NPCR); the National Cancer Institute’s Surveillance, Epidemiology and End Results Program (NCI SEER); the National Cancer Registrars Association (NCRA); and the North American Association of Central Cancer Registries (NAACCR .

The CS Task Force’s assignment was to develop a coding system between the American Joint Committee on Cancer’s (AJCC) TNM staging system, SEER Extent of Disease (EOD) and the SEER Summary Staging system that would eliminate duplicate data collection by registrars. Collaborative Stage version 01.01.00 was released August 11, 2004 and was effective for all new cancers diagnosed January 1, 2004 and after. The system was comprised of 25 data items that described the extent of disease at the time of diagnosis as well as site‐specific factors (prognostic indicators). Most of these data items are similar to the Extent of Disease (EOD) data items that have been collected by SEER registrars for many years. New data items were added: evaluation fields that described the basis and sources for the collected data (i.e. clinical vs. surgical) and site‐specific factors. These fields were added to assist in deriving the final stage grouping and the computer algorithms that classify each case in multiple staging systems.

The CSv2 Work Group was formed to make changes to the Collaborative Stage System based on

new staging criteria from the AJCC Cancer Staging Manual, 7th Edition. This included the additions of prognostic indicators and tumor markers that were felt to be clinically significant. Major changes included the addition of several new schemas and revisions to most schemas, including the addition of up to 19 new site specific factors. Changes to the schemas included, new codes, changing the hierarchy of codes, note revisions and the rewriting of Part I, including a more expanded section on the Site Specific Factors ( SSFs).

In 2010, the CSv2 Field Study Team developed coding practice studies to evaluate the cancer registrar’s ability to code the CSv2 items consistently and to evaluate inconsistencies so that improvements could be made to codes, coding structures, notes, and instructions. The primary sites targeted were bladder, breast, colon, corpus, esophagus GE junction, GIST stomach, kidney parenchyma, lung, melanoma skin, merkel cell, NET colon, thyroid, testis and tonsil. As of September 2012, the coding practice studies have been completed for all schemas. Due to the major changes made in CS for AJCC 7th edition, the CSv2 Field Study Group’s goal was to have 2000 registrars participate in this study. There were a total of 1,040 participants. To reach this number of participants, it required the cooperation of all the organizations involved in cancer registration, including, but not limited to,

American Joint Committee on Cancer (AJCC)/Commission on Cancer (CoC)

Canadian Partnership Against Cancer (CPAC)

Canadian Council of Cancer Registries (CCCR)


Pg. 4 of 32

Centers for Disease Control (CDC) National Program of Cancer Registries (NPCR)

National Cancer Institute (NCI) Surveillance, Epidemiology, and End Results (SEER)

National Cancer Registrars Association (NCRA)

North American Association of Central Cancer Registries (NAACCR The purpose of the involvement of all these organizations was to encourage the registrars associated with their organizations to participate. The high number of participating registrars in 2011 provides a greater opportunity to assess how the registry community is handling CSv2.

STUDY OBJECTIVE The 2011 CS Reliability study was conducted to: 1. Assess the accuracy and consistency with which registrars are able to use CSv2, version 0203. 2. Coordinate efforts among the organizations to use this information to refine CSv2 rules,

documentation and training for all organizations. 3. Identify clarifications and changes for future versions of CSv2. 4. Identify future quality improvement opportunities

INTRODUCTION

A reliability study is a quality improvement process designed to assess the proficiency of central and hospital cancer registrars and to measure consistency in the use of codes and coding rules across a program. In a reliability study, all participants code information under similar conditions from the same medical reports using the same references.

A quality improvement process uses reliability studies to understand variability in participants’ coding compared to the expected or preferred answers (based on expert panel’s consensus answers). Unexpected answers for this study were reviewed using rationale from comments on the website from the participants and discussion with the expert panel. Rationales for all answers were provided. The 2011 CSv2 Reliability Study is an assessment of two major factors:

the revision of rules and clarifications as a result of the 2005 CS Study Assess if the registrar’s comprehension and knowledge of CS staging has improved with two

additional years of use. Based on the findings of this study, recommendations will be made for changes and/or

clarifications to the Collaborative Staging Manual Data Collection System‐Version 2. Priority topics for educational interventions will also be developed.

PRELIMINARY STEPS In preparation for the CSv2 2011 Reliability Study, the results of previous NCI SEER Re‐evaluation

Reliability Studies were reviewed to determine the areas that were identified as needing documentation changes or educational interventions.

Criteria were developed for each cancer site based on problems found in previous reliability studies

as well as new and complex schemas that were introduced in CSv2. Based on these criteria, a call for cases was done using the CoC listserve.


Pg. 5 of 32

The Site Group Teams (Appendix J) reviewed the cases. Each of the four Site Group Teams was assigned specific cancer sites. The Site Group Teams prepared the cases for transcription for the web and determined the preferred answer for each case.

The forty‐four cases were converted into a Microsoft Word standardized format by a transcriptionist who contracts with NCI SEER. Information Management Services Inc. (IMS), a biomedical computing contractor with NCI SEER, converted the cases to HTML format for placement on the study website.

The preferred answers with rationales were posted by IMS to the study website.

STUDY PARTICIPANTS

Each participating organization was asked to provide a representative to coordinate participation from their members and to answer questions. Notices of the study were sent out to the cancer registry community via the CoC flash, CDC NPCR notices, the NCRA “Connection”, NCI SEER listserve, and the NAACCR listserve. The contact for Canada sent notices to each of the provinces. The study was open for seven weeks to allow ample time for participation from the registry community. The result was the highest number of participants (1040) of the three reliability studies (Table 2). The majority of the U.S. participants were from CoC hospitals (66%) reflecting the fact that most of the coding of cancer data in the U.S. occurs in hospitals. The participants were well educated and experienced with 88% having Certified Tumor Registrar (CTR) status and the majority having more than six years’ experience with CS and more than 1 year with CSv2. Since Collaborative Stage was initiated in 2004 and CSv2 effective for 2010 diagnoses, this was the maximum experience possible.

Most study registrants (81%) completed at least one of the cases that they had been assigned to code, as shown in Figure 1. Each participant received an initial assignment of 10 cases, and upon request, the participant was assigned 10 additional cases, for a maximum of 20 cases. Twenty‐eight percent of participants completed the full 20 cases, and an additional 50% of participants completed 10 cases or more. Just 3% of participants did not fully complete the initial 10 cases. The normal work responsibilities of participants included case finding, abstracting and follow‐up (Figure 2). Over 40% of participants had also been involved in data analysis, data editing, and other QC activities. Figure 3 presents the distribution of study participant by years of experience in the cancer registration field. Close to one‐third of participants reported 15 years of experience or more, with another 46% reporting at least five years of experience. Just 12% of participants reported less than five years of cancer registry experience.


Pg. 6 of 32

Table 1.

* Non‐CoC hospitals, contractors, registry services companies and national standard setters

Table 1: Characteristics of 2011 Study Participants

N %

Total 1040 100

Type of Organization

Canada 42 4%

CoC Hospitals 686 66%

Central Registries 218 21%

Other* 94 9%

Credential

CTR 915 88%

Non‐CTR 125 12%

Years of Experience CS

Less than 1 year 62 6%

2 to 6 years 302 29%

More than 6 years 676 65%

Years of Experience CSv2

Less than 6 months 83 8%

6 months to 1 year 312 30%

More than 1 year 645 62%


Pg. 7 of 32

FIGURE 1. DISTRIBUTION OF STUDY PARTICIPANTS BY THE NUMBER OF CASES COMPLETED

FIGURE 2. MOST COMMON WORK RESPONSIBILITIES OF STUDY PARTICIPANTS

43.8%

44.4%

49.3%

50.7%

64.2%

85.9%

0% 20% 40% 60% 80% 100%

Data analysis

Editing/case consolidation

Quality control

Follow-up

Casefinding

Abstracting

Top 5 most frequent answers


Pg. 8 of 32

FIGURE 3. DISTRIBUTION OF PARTICIPANTS BY YEARS OF EXPERIENCE IN CANCER REGISTRATION (N=1040)

15+ yrs.32%

10-14 yrs.8%5-9 yrs.

38%

0-4 yrs.12%

Unknown10%

15+ yrs.

10-14 yrs.

5-9 yrs.

0-4 yrs.

Unknown

DATA ITEMS

The data items in the study are a combination of long standing core data items such as tumor size and extension (Table 3) and newer fields, site specific factors (SSFs) that vary by schema. Examples of SSFs include Prostatic Specific Antigen (PSA) for prostate and Estrogen Receptor (ER) status for breast cancer and are listed in Appendix E. The data items included in the study are those required by SEER and CoC for CSv2. Some SSFs were not required by the other organizations (mostly NPCR registries and non‐CoC hospitals); however, all participants completed the same set of data items for each case. Participants had the option of completing non‐required SSFs questions for this study and these results are not included in the final analysis. The number of SSFs varies by schema with breast having 24 SSFs compared to only 2 for lung.

Table 2: Core Data Table 2: CS Data Items

Primary site CS Tumor Size

Histology CS Extension

Behavior code CS TS/Ext Eval

Laterality CS Lymph Nodes

Age (Thyroid only) CS Nodes Eval

LVI (Testis only) Reg LN Pos

Grade (Esophagus only) Reg LN Exam

CS Mets at Dx

CS Mets at DX – Bone

CS Mets at Dx – Brain

CS Mets at DX Liver

CS Mets at DX Lung

CS Mets Eval


Pg. 9 of 32

RECONCILIATION

Reconciliation provides an opportunity for registrars to provide their coding rationale, especially when they disagree with the preferred answer. This gives the study developers a chance to see where some of the problems within CS exist and provides educational opportunities to address those problems. Due to the number of sites/schemas involved in this study, the traditional format for doing reconciliation was changed. In the past, SEER distributed the data items eligible for reconciliation and then set up conference calls with their registries and other interested parties to discuss the data items. Changes were made based on these interactions. During the planning process for this study, it was determined that this way of doing reconciliation was not feasible and an alternate method was developed. The reconciliation process used for this study was an online web application open 24 hours a day for 2 weeks. This process gave more people the opportunity to participate. Of the 1035 data items collected in the study, 439 were eligible for reconciliation. Below is the distribution of the eligible data items for each of the groups. Table 3.

Group Total Data Items Core Data Items SSFs Total % of Total

1 262 65 81 146 55.7%

2 265 43 67 110 41.5%

3 263 35 59 94 35.7%

4 245 28 61 89 36.3%

Total 1035 171 268 439 25.9%

Reconciliation was open for two weeks and over 7,200 responses were entered by 284 (approximately 27% of participants) registrars into the online reconciliation. The comments were grouped by the type of data item. The Non CS data items include: primary site, histology, laterality, age (thyroid only) and LVI (testis only). The CS Core data items include: Tumor Size, Extension, TS/Ext Eval, Lymph nodes, LN eval, Reg nodes positive, Reg nodes examined Mets at Dx and Mets Eval. The Site specific factors are listed in Appendix E. The optional SSFs, which are not required by any of the standard setters, are not included in the analysis. Table 4.

Types of Comments Received

Type of Data Items Comments Received % of Total Comments

Non CS Data Items 596 8.2%

CS Core Data Items 2895 39.8%

Required SSFs 2731 37.6%

Optional SSFs 1047 14.4%

Total 7269 100%


Pg. 10 of 32

After the responses were received, the four teams reviewed comments from the registrars. The cases were reviewed once again with the registrar’s comments. Based on this, 15 answers were changed, rationale was updated and/or multiple answers were allowed. The experts identified at the beginning of the study were contacted and asked to help with many of these answers. The table below summarizes the changed answers based on reconciliation. Table 5. Reconciliation Results

Group Case Data item Preferred Answer

Final Answer(s)

Reason for change

1 Breast 1 SSFs 12, 13, 14, 17, 18, 22, 23

999 999,998 No documentation. Preferred answer unknown (999). Test not done (998) also accepted

1 Breast 2 SSFs: 10, 11, 12, 13, 14, 17, 18, 22,

23


1 Corpus Reg Nodes Examined

16 16, 18 Due to differences in path report and CAP report, both node counts accepted

1 Lung 1 Reg nodes positive

02 00 Further evaluation determined that these nodes were fine needle aspirations and did not qualify for nodes examined

1 Lung 1 Reg nodes examined

02 95

1 Lung 1 Mets at Dx‐Lung

1 0 Pleural effusion in contralateral lung coded as mets. According to Part I, Section 1, this should not be coded as lung mets

1 Merkel Cell Skin

SSF 17 090 030 No mention of matted or fixed lymph nodes which is needed to code 030

1 NET Colon SSF 2 999 000 No regional nodes seen on CT

1 NET Colon SSFs 16, 17 999 999,998 No documentation. Preferred answer unknown (999). Test not done (998) also accepted

1 Testis SSF 8 997 997, 001 Code 001 added due to minimal information

1 Testis SSF 13 992 997 Need to code values after orchiectomy and before additional treatment

1 Testis SSF 15 000 990

1 Thyroid Age 052 052, 52 Both formats accepted

2 Breast 1 SSF 10, 11, 12, 13, 14,

999 999,998 No documentation. Preferred answer unknown


Pg. 11 of 32


Final Answer(s)

Reason for change

17, 18, 22, 23

(999). Test not done (998) also accepted

2 Breast 2 SSFs 10, 11, 14, 17, 18, 22, 23


2 Esophagus EG Junc

CS Mets at Dx‐Liver

0 1 Positive mets on peritoneal biopsy

2 Lung 2 Extension 740 700 Aorta not involved, para‐aorta is. Code to mediastinal involvement

2 Lung 2 SSF 2 040 999 Elastic layer not evaluated

2 Merkel Cell Skin

Extension 400 400, 600 Due to minimal information, 2 codes accepted

2 Merkel Cell Skin

SSF 16 300

990 Cannot determine size based on a percentage given

2 NET Colon SSFs 16, 17 999 999,998 No documentation. Preferred answer unknown (999). Test not done (998) also accepted

2 NET Colon Mets at Dx‐Bone

00 00, 99 No mention of bone mets

3 Colon 1 SSFs 1, 3, 7, 9, 10


3 Colon 1 SSF 2 999 000 Negative CT scan. Enough to code no clinical LN involvement

3 Colon 2 SSFs 7, 9, 10 999 999,998 No documentation. Preferred answer unknown (999). Test not done (998) also accepted

3 GIST Stomach

SSFs 8, 9 999 999,998 No documentation. Preferred answer unknown (999). Test not done (998) also accepted

3 Kidney SSF 3 999 000 Adrenal gland involvement not mentioned, can code none

3 Ovary Mets Eval 1 0

4 Colon 1 SSFs 1, 3, 10 999 999,998 No documentation. Preferred answer unknown (999). Test not done (998) also accepted


Pg. 12 of 32


Final Answer(s)

Reason for change

4 Colon 2 SSFs 1, 3, 9, 10


4 Colon 2 SSF 2 000 999 Only clinical workup is chest xray, not enough to code negative clinical LN’s

4 GIST Stomach

SSFs 8, 9 999 999,998 No documentation. Preferred answer unknown (999). Test not done (998) also accepted

4 Kidney Lymph Nodes Eval

1 0

4 Melanoma Skin

SSFs 4, 5, 6 999 999,998 No documentation. Preferred answer unknown (999). Test not done (998) also accepted

4 Ovary Mets Eval 1 0

MAJOR, MINOR AND UNKNOWN TO KNOWN (Unk2k) ERRORS

There are three error classifications: major, minor, and unknown‐to‐known. (See Appendix C: Definition of Errors) Major errors are those that affect data analysis (staging) or incidence, specifically for Collaborative Staging items (major errors resulted in changes within Tumor, Nodes or Metastasis (TNM)). Minor errors show a lack of specificity that would have no important effect on data analysis or changes within TNM. Unknown‐to‐known errors occur when the participant uses a code meaning “unknown” when more specific information is available in the chart, or when a default code should have been used. Unknown‐to‐known errors may or may not affect analysis. If they do affect analysis, they are counted as major errors. The graph below shows the distribution of the major errors by CS Core Data Items (Tumor Size, Extension, TS/Ext Eval, Lymph Nodes, Regional nodes positive, Regional nodes examined, Lymph Node Eval, Mets at Dx and Mets Eval) and Stage Related SSFs. (For a site by site, item by item, case by case analysis of frequencies, please see Appendix E: Site, Data, Case response listing.)


Pg. 13 of 32

Table 6.

Accuracy Rate Goals for 2011 Study (Based on % of cases without Major Errors) Cells highlighted met the Accuracy Goal

Schema Stage Related Core Items Stage Related SSFs

Goal Actual Goal Actual

Bladder 95 93.0% 95 n/a

Brain 95 n/a 95 n/a

Breast 95 96.8% 95 90.8%

Colon 95 97.2% 95 98.6%

Corpus 95 97.5% 95 n/a

EGJ 95 86.3% 95 90.7%

GIST St. 95 97.3% 95 53.8%

Lung 95 79.7% 95 77.8%

Kidney 95 97.4% 95

Melanoma Skin 95 84.2% 95 78.5%

Merkel Cell Skin 95 96.3% 95 66.5%

NET Colon 95 84.4% 95 99.7%

Pharyngeal Tonsil 95 96.2% 95 88.1%

Ovary 95 91.7% 95 n/a

Prostate 95 95.4% 95 89.4%

Testis 95 86.8% 95 68.5%

Thyroid 95 95.7% 95 n/a

Below is a distribution of the types of answers for each of the major groups of cases in the study. Brain is not included since it is not staged.


Pg. 14 of 32

Figure 4: Distribution of Correct, Major, Minor and Unk2‐k answers for Breast, Colon, Lung and Prostate for CS Core Data Items

Figure 5: Distribution of Correct, Major, Minor and Unk2‐k answers for Corpus, GIST Stomach, Merkel Cell Skin, NET Colon and Testis for CS Core Data Items


Pg. 15 of 32

Figure 6: Distribution of Correct, Major, Minor and Unk2‐k answers for Corpus, GIST Stomach, Merkel Cell Skin, NET Colon, Thyroid and Testis for CS Core Data Items

Figure 7: Distribution of Correct, Major, Minor and Unk2‐k answers for Bladder, Esophagus GE Junc, Kidney, Melanoma Skin, Ovary and Pharyngeal Tonsil for CS Core Data Items


Pg. 16 of 32

Figure 8: Distribution of Correct, Major, Minor and Unk2k answers for Breast SSFs 9, 11, 13, 15, 16. Below is the distribution for the HER2NEU interpretation and summary SSFs. The major errors are defined by registrar’s using a code of 988, which is not applicable. For these SSFs, two answers of 998 (test not done) and 999 (unknown) were accepted for the SSFs where the answer was not documented in the record. SSF 16 is the only data item in the entire study that had a 100% agreement with the final answer.

Figure 9: Distribution of Correct, Major, Minor and Unk2k answers for Prostate SSFs 3, 7, 8, 9, 10, and 11. Below is the distribution for pathologic extension for Prostate and the Gleason SSFs. Major errors for SSF 3, 8 and 10 are due to these factors being used for stage. SSFs 7 and 9 major errors are defined by registrars coding 988, which is not applicable. Since these are required SSFs, code 988 is not allowed.

For a complete listing of schemas and SSFs by type of answer (correct, major, minor and unk2k), see Appendix E


Pg. 17 of 32

ANALYSIS OF FACTORS AFFECTING THE ACCURACY RATE As shown in the Major, Minor, and Unknown–to‐Known Errors analysis chapter, the accuracy of coding CS v2 data elements varied significantly by CS schema and data element. Numerous factors could explain these results. Some of the factors were related to the characteristics of the cases presented for coding, whereas others were related to the attributes of study participants. This section of the report focuses on the presentation of observed accuracy rate and the association of the rate with the characteristics of study participants. As with any study involving medical coding, it was expected that the accuracy would be correlated with the complexity of coding rules and the number of valid values that the system will accept for each data element. For example, data elements such as tumor behavior and laterality are generally coded more accurately than histology, because there are fewer values that a registrar has to select from as compared with the multitude of values available for coding primary site or histology. Meanwhile, primary site and histology are more important for CS coding because miscoding these data elements can result in the selection of an incorrect CS schema, with catastrophic consequences regarding the accuracy of coding all CS data elements for that given case. Table 7 presents the proportion of accurate coding for primary site, histology, behavior, and laterality for all CS schemas included in the study. The results are aggregated by CS schema, in the sense that the proportions shown for a given schema have been calculated based on answers from all participants who attempted to code a case (e.g. 1,399 answer for breast cases, 712 for bladder cases, etc.). As expected, the behavior and laterality are coded correctly for most sites, with the exception of laterality of thyroid and pharyngeal tonsil cases. Meanwhile, there is a huge variation with respect to the accuracy of primary site, ranging from perfect results for thyroid, kidney, ovary, and pharyngeal tonsil, to very poor results for bladder, breast, testis and GIST stomach. The perfect results are explained by the fact that there is a unique valid value for coding thyroid, ovarian, kidney parenchyma and pharyngeal tonsil tumors. Similarly, the histologic accuracy was perfect for Merkel cell CS schema, which is defined by a single value for histology. However, the results showed low accuracy for the coding of the histology of thyroid, breast, and testicular tumors. This finding is particularly important because errors related to histology for these sites can result in assigning a stage to tumors that are not covered by AJCC staging.


Pg. 18 of 32

Table 7. Total Participants and Proportion of the correct answer for Primary site, Histology, Behavior and Laterality by CS schema.

CS Schema Primary site Histology Behavior Laterality

# % # % # % # %

Bladder 712 30.2 712 85.0 712 96.1 712 95.1

Breast 1,399 69.7 1399 69.2 1,399 99.6 1399 98.6

Corpus carcinoma

691 82.2 691 95.1 691 99.7 691 99.1

Brain 703 93.0 703 96.7 703 99.9 703 79.7

Colon 1,381 97.0 1,381 87.5 1,381 99.1 1,381 97.0

Net Colon 683 97.5 683 98.5 683 99.9 683 99.1

Esophagus GE 694 97.1 694 97.3 694 99.7 694 99.9

Lung 1,374 85.8 1,374 91.9 1374 99.9 1374 96.5

Thyroid 332 100.0 332 25.6 332 99.7 332 NA

Testis 680 50.7 680 45.1 680 99.1 680 97.1

Prostate 1,341 100.0 1,341 66.3 1,341 99.9 1,341 96.9

Melanoma 674 88.6 674 98.7 674 100 674 94.2

Merkel Cell 686 97.7 686 100 686 100 686 94.3

Kidney Parenchyma

678 100.0 678 73.0 678 99.7 678 98.8

GIST Stomach 682 63.9 682 99.1 682 99.3 682 99.9

Ovary 672 100.0 672 95.7 672 99.9 672 98.2

Pharyngeal tonsil

337 100.0 337 99.4 337 99.7 337 NA

The next three tables present the accuracy of selecting the preferred code value for all core CS data elements. These tables include the values for three data elements that are crucial for determination of both SEER Summary Stage and AJCC Stage. The three data elements are CS Extension, CS Lymph Nodes and CS Mets at Diagnosis. Table 8 shows the accuracy of coding tumor size, the direct extension of tumor, and the method for evaluating the size and the extension. In general, the tumor size is coded more accurately than the tumor extension. Except for skin melanomas, the accuracy of coding tumor extension was poor. Very low accuracy of coding tumor extension was observed for some of the most frequent tumor sites, such as bladder, lung and colon. While the accuracy of coding the extension of skin melanomas was almost perfect, the accuracy of coding tumor size was very low (48.5%), second worst after the accuracy of coding the size of brain tumors (47.9%). In general, the accuracy of size/extension evaluation was somewhat better than the accuracy for size or for extension. Nevertheless, there was no general pattern. For example, brain has low accuracy for tumor size (47.9%) and extension (82.9%) but perfect accuracy for evaluation, while for the pharyngeal tonsil schema, the accuracy of coding size was very good ( 95.3%) and the accuracy of coding the evaluation method was poor ( 42.4%). Table 9 presents the accuracy of coding data elements that describe the dissemination of tumors to regional lymph nodes. This group of data elements include: the CS Lymph Nodes, the Lymph Nodes Evaluation, Regional


Pg. 19 of 32

Lymph Nodes Examined, and the Regional Lymph Nodes Positive. With few exceptions, the CS Lymph Nodes was coded more accurately than the CS Extension and CS Tumor Size. The exceptions are breast, skin melanomas, thyroid and pharyngeal tonsils tumors. The perfect accuracy recorded for brain tumors is explained by the absence of regional lymphatic dissemination in brain tumors. Except for brain, the CS schema with best accuracy for lymph nodes was Prostate. At the other end of the accuracy spectrum, low accuracy was noted for pharyngeal tonsils, colon tumors, and skin melanomas. Also, the accuracy was generally better for the two data elements that pre‐dated the CS schema: regional nodes examined and regional nodes positive. Accuracy of coding CS data elements that describe distant metastases are shown in table 10. In general, the accuracy of coding metastases is better than the accuracy of coding regional lymph nodes involvement, local extension or tumor size. Noticeable exceptions have been observed for coding the metastasis of lung tumors (43.9%) and of the NET colon tumors (20.5%). The CS schemas of tumors with highest incidence in population, which are breast and prostate tumors, both showed very good accuracy for all CS data elements. The four data elements that have been required for reporting more recently, Mets‐Lung, Mets‐Brain, Mets‐Liver and Mets‐Bone, while apparently are easier to code, have not been coded more accurately than the CS Mets at Dx. Less accurate coding in this group of data elements was observed for Mets at Dx evaluation, with the accuracy of coding Corpus carcinoma metastases below 20% and the accuracy of esophagus GE tumors at just 38.2%.

Table 8. Total Participants and Proportion of the correct answer for Tumor Size, Extension, and Size/extension Evaluation by CS schema.

CS Schema Tumor Size

Extension

Size/extension Evaluation

# % # % # % Bladder 712 80.2 712 38.6 712 82.0

Breast 1,399 90.9 1,399 89.4 1,399 98.9

Corpus carcinoma 691 90.6 691 77.0 691 99.3

Brain 703 47.9 703 82.9 703 100.0

Colon 1,381 87.5 1,381 44.7 1,381 97.2

Net Colon 683 74.1 683 25.9 683 99.0

Esophagus GE 694 64.7 694 56.9 694 34.9

Lung 1,374 73.7 1,374 35.1 1,374 46.1

Thyroid 332 89.8 332 88.0 332 98.8

Testis 680 93.2 680 60.0 680 98.1

Prostate 1,341 86.3 1,341 51.8 1,341 72.6

Melanoma 674 48.5 674 96.1 674 97.5

Merkel Cell 686 84.5 686 56.1 686 63.3

Kidney Parenchyma

678 97.1 678 88.6 678 99.1

GIST Stomach 682 97.1 682 51.3 682 99.0

Ovary 672 75.6 672 74.4 672 99.3

Pharyngeal tonsil 337 95.3 337 88.4 337 42.4


Pg. 20 of 32

Table 9. Total Participants and Proportion of the Accurate Answers for Lymph Nodes, Lymph Nodes Evaluation, Lymph Nodes examined, and Lymph Nodes Positive, by CS Schema.

CS Schema Lymph Nodes

Lymph Nodes Evaluation

Lymph Nodes Examined

Lymph Nodes Positive

# % # % # % # %Bladder 712 78.7 712 85.5 712 97.8 712 96.2

Breast 1,399 60.3 1,399 98.5 1,399 85.9 1,399 81.5

Corpus carcinoma

691 81.2 691 98.6 691 98.1 691 99.1

Brain 703 100.0 703 100.0 703 100.0 703 100.0

Colon 1,381 59.8 1,381 85.3 1,381 96.3 1,381 97.9

Net Colon 683 50.7 683 93.9 683 92.2 683 94.0

Esophagus GE 694 60.2 694 41.4 694 81.3 694 87.0

Lung 1,374 68.7 1,374 73.2 1,374 74.8 1,374 73.7

Thyroid 332 69.9 332 77.4 332 95.5 332 94.6

Testis 680 76.6 680 54.9 680 47.1 680 50.1

Prostate 1,341 92.5 1,341 92.5 1,341 80.1 1,341 94.7

Melanoma 674 51.5 674 81.0 674 65.0 674 65.6

Merkel Cell 686 89.7 686 95.8 686 96.5 686 95.5

Kidney Parenchyma

678 89.5 678 84.7 678 97.8 678 96.6

GIST Stomach 682 89.4 682 58.2 682 96.6 682 96.5

Ovary 672 91.1 672 68.2 672 83.9 672 86.5

Pharyngeal tonsil

337 46.0 337 90.5 337 93.5 337 89.9


Pg. 21 of 32

Table 10. Total Participants and Proportion of the correct answers for CS Metastases‐related data elements by CS schema.

CS Schema Mets at Dx

Mets at DX Evaluation

Mets at Dx‐Bone

Mets at Dx‐ Brain

Mets at Dx‐ Liver

Mets at Dx‐ Lung

# % # % # % # % # % # % Bladder 712 80.3 712 87.8 712 81.0 712 86.9 712 87.5 712 87.6

Breast 1,399 93.9 1,399 93.3 1,399 94.1 1,399 92.6 1,399 94.1 1,399 93.9

Corpus carcinoma

691 92.9 691 14.8 691 91.8 691 91.2 691 92.0 691 92.0

Brain 703 92.5 703 100.0 703 87.3 703 86.3 703 88.5 703 88.2

Colon 1,381 92.1 1,381 63.2 1,381 95.3 1,381 94.9 1,381 97.5 1,381 95.4

Net Colon 683 20.5 683 63.1 683 94.1 683 90.9 683 94.9 683 93.9

Esophagus GE 694 85.2 694 38.2 694 95.8 694 95.0 694 86.0 694 97.4

Lung 1,374 43.9 1,374 82.2 1,374 93.6 1,374 92.6 1,374 94.8 1,374 85.7

Thyroid 332 82.8 332 82.2 332 82.2 332 82.2 332 83.1 332 82.5

Testis 680 96.0 680 93.2 680 97.5 680 97.9 680 98.1 680 99.0

Prostate 1,341 97.9 1,341 96.6 1,341 98.4 1,341 95.5 1,341 96.4 1,341 96.3

Melanoma 674 94.4 674 93.5 674 94.1 674 93.8 674 94.4 674 96.0

Merkel Cell 686 90.4 686 96.1 686 97.1 686 96.6 686 97.5 686 94.0

Kidney Parenchyma

678 97.2 678 88.5 678 95.6 678 94.8 678 97.1 678 97.9

GIST Stomach 682 98.1 682 54.4 682 96.8 682 95.9 682 97.5 682 98.1

Ovary 672 92.4 672 70.5 672 94.2 672 94.3 672 96.4 672 95.1

Pharyngeal tonsil

337 90.5 337 92.0 337 91.1 337 91.4 337 89.9 337 89.9

Core CS data elements, and certain site‐specific data elements, are used to derive the AJCC Group Stage and the Summary Stage. Table 11 presents the accuracy of stage value that was derived for each case based on the set of answers that the participants returned for the case. For the purpose of calculating the proportion accurate stage, the stage derived for each case completed by a participant was compared with the preferred stage derived from the set of final preferred answers adopted by the study team. Table 11 presents the accuracy calculated independently for Summary Stage 1977, Summary State 2000, AJCC Stage Group edition 6 and AJCC stage Group edition 7. According to the table, the Accuracy of AJCC 7 Stage for breast cases was 83.1%. This can be interpreted that for 83.1 percent of the breast cases coded by study participants, the AJCC stage calculated (derived) from the answers received from participants was correct. For the remaining 16.9% of the cases, the combination of CS codes selected by participants resulted in the inaccurate staging of the case.


Pg. 22 of 32

Some of the most accurate answers across the four staging systems were obtained for kidney parenchyma, ovary, breast, brain, lung and the GIST tumors of stomach. High proportions of inaccurate stage have been observed for skin melanomas, neuroendocrine tumors of colon, pharyngeal tonsils, esophagus and prostate tumors. In general there have been fewer errors for the SEER Summary Stage system than for the AJCC system. Also, AJCC Stage Group edition 7 performed better than edition 6, although differences are minimal. As someone would expect, the more data elements considered for the derivation of stage, the higher the likelihood of disagreement with the preferred stage. Based on table 11, it can be inferred that Summary Stage 2000 and AJCC Stage Group 7th edition are probably the best global indicators of coding accuracy. Accordingly, the analytic team focused the rest of analysis on finding predictors of these to global coding outcome indicators.

Table 11. Participants Count and the Proportion of Correct Answers by CS Schema and Staging System

CS Schema AJCC 6 Stage AJCC 7 Stage Summary Stage 1977

Summary Stage 2000

# % # % # % # %

Bladder 712 61.7 712 62.5 712 81.3 712 81.3

Breast 1,384 79.7 1,383 83.1 1,397 96.6 1,397 96.6

Corpus carcinoma

689 83.9 691 92.3 691 88.0 691 88.0

Brain 702 NA 702 NA 702 86.6 702 86.6

Colon 1,381 90.2 1,373 89.7 1,381 78.2 1,381 79.0

Net Colon 683 31.3 683 64.6 683 56.8 683 56.8

Esophagus GE 690 72.2 679 64.7 691 88.1 691 87.7

Lung 1,373 87.8 1,371 89.1 1,374 91.3 1,372 90.7

Thyroid 331 69.5 331 69.8 332 100.0 332 100.0

Testis 636 31.6 636 32.1 671 89.1 671 90.0

Prostate 1,323 78.9 1,341 69.0 1,341 86.8 1,341 88.5

Melanoma 474 44.3 474 45.4 672 50.6 672 50.4

Merkel Cell 686 87.2 562 73.5 686 87.8 686 90.4

Kidney Parenchyma

677 86.1 677 96.5 677 96.3 677 96.3

GIST Stomach 682 NA 682 51.0 682 90.6 682 90.6

Ovary 672 93.6 672 71.9 672 95.2 672 92.9

Pharyngeal tonsil

322 78.3 322 85.1 325 67.1 325 58.2


Pg. 23 of 32

Table 12. Comparison of Proportion Accurate Stage by Certification Status, for Certain Staging Systems and CS Schemas.

CS v2 Schema Summary Stage 2000 AJCC Stage Group 7th edition

CTR Non‐CTR CTR Non‐CTR

Na % accurateb

Na % accurateb

pc Na % accurateb

Na % accurateb

pc

Bladder 632 81.5 80 80.0 0.75 632 62.5 80 62.5 1.00

Breast 1,242 96.9 157 93.0 0.01d

1,242 82.9 157 76.4 0.05d

Corpus Carcinoma

614 87.8 77 89.6 0.64 614 91.7 77 97.4 0.08

Brain 634 87.5 69 76.8 0.01d

NA NA NA NA

Colon 1,245 79.1 136 77.9 0.75 1,245 89.4 136 86.8 0.35

Net Colon 608 56.1 75 62.7 0.28 608 65.1 75 60.0 0.38

Esophagus GE 615 88.0 79 82.3 0.15 615 64.9 79 50.6 0.01d

Lung 1,223 90.9 151 87.4 0.16 1,223 89.4 151 85.4 0.15

Thyroid 300 100.0 32 100.0 1.00 300 69.7 32 68.8 0.91

Testis 607 88.6 73 90.4 0.65 607 31.3 73 19.2 0.03d

Prostate 1,205 89.2 136 82.3 0.02d

1,205 70.5 136 55.9 <0.01d

Melanoma 606 50.3 68 50.0 0.96 606 32.2 68 29.4 0.64

Merkel Cell 611 90.7 75 88.0 0.46 611 60.6 75 57.3 0.59

GIST Stomach 614 90.9 68 88.2 0.48 614 51.1 68 50.0 0.86

Ovary 604 92.9 68 92.7 0.94 604 73.3 68 58.8 0.01d

Pharyngeal tonsil

305 56.7 32 50.0 0.47 305 82.6 32 68.8 0.06

Kidney Parenchyma

610 96.7 68 91.1 0.02e

610 97.1 68 89.7 <0.01d

a=number of distinct set of answers collected from participants and used for stage derivation. b=proportion correct derived stage (at the sub stage level). c=p‐value associated with Pearson chi‐square statistic (for the association between correct stage and CTR/Non‐CTR status). d=statistically significant


Pg. 24 of 32

Table 13. Comparison of Proportion Accurate Stage by CS Coding Experience, for Certain Staging Systems and CS Schemas.


>= 5 yrs. CS experience

<5 yrs. CS experience Pd

>= 5 yrs. CS experience

<5 yrs. CS experience Pd

Nb % accuratec

Nb % accuratec

Nb % accuratec

Nb % accuratec

Bladder 553 81.6 159 80.5 .76 553 62.8 159 61.6 .80

Breast 1,088 96.8 311 95.2 .18 1,088 83.7 311 76.5 <.01e

Corpus Carcinoma

538 89.0 153 84.3 .11 538 92.4 153 92.2 .93

Brain 559 86.2 144 87.5 .69 NA NA NA NA

Colon 1,098 79.7 283 76.3 .22 1,098 90.6 283 83.4 <.01e

Net Colon 532 54.9 151 63.6 .06 532 66.4 151 58.3 .07

Esophagus GE 550 88.2 144 84.0 .18 550 65.1 144 56.3 .05e

Lung 1,071 91.0 303 88.8 .24 1,071 89.4 303 87.5 .35

Thyroid 256 100.0 76 100.0 1.00 256 71.9 76 61.8 .10

Testis 530 89.3 150 87.3 .51 530 31.1 150 26.0 .23

Prostate 1,059 89.2 282 85.8 .11 1,059 69.2 282 68.1 .72

Melanoma 533 51.4 141 46.1 .26 533 32.5 141 29.8 .55

Merkel Cell 535 91.4 151 86.8 .09 535 61.1 151 57.0 .36

GIST Stomach 541 91.3 141 87.9 .22 541 51.0 141 51.2 .99

Ovary 531 93.0 141 92.2 .73 531 73.8 141 64.5 .03e

Pharyngeal tonsil 265 56.6 72 54.2 .71 265 83.8 72 72.2 .03e

Kidney Parenchyma

537 96.3 141 95.7 .77 537 96.5 141 95.7 .69

a=experience with Collaborative Staging system categories: “5 or more years of experience” versus “less than 5 years of experience”. b=number of distinct set of answers collected from participants and used for stage derivation. c=proportion correct derived stage (at the sub stage level). d=p‐value associated with Pearson chi‐square statistic (for the association between correct stage and Collaborative Staging years of experience). e=statistically significant


Pg. 25 of 32

Table 14. Comparison of Proportion Accurate Stage by CS v2 Coding Experience, for Certain Staging Systems and CS Schemas.


>= 12 mos. CS v2 experience

< 12 mos. CS v2 experience Pd

>= 12 mos. CS v2 experience

< 12 mos. CS v2 experience Pd

Nb % accuratec

Nb % accuratec

Nb % accuratec

Nb % accuratec

Bladder 436 83.5 276 78.0 .06 436 66.1 276 56.9 .01e

Breast 856 97.0 543 95.6 .17 856 83.0 543 80.9 .32

Corpus Carcinoma 423 89.1 268 86.2 .25 423 92.7 268 91.8 .67

Brain 445 88.8 258 82.6 .02e NA NA NA NA

Colon 876 79.2 505 78.6 .79 876 91.4 505 85.2 <.01e

Net Colon 417 57.6 266 55.6 .62 417 66.2 266 62.0 .27

Esophagus GE 420 87.9 274 86.5 .60 420 65.5 274 59.9 .13

Lung 841 90.4 533 90.8 .79 841 89.1 533 88.7 .85

Thyroid 204 100.0 128 100.0 1.00 204 73.5 128 63.3 .05e

Testis 415 89.4 265 87.9 .55 415 30.4 265 29.4 .80

Prostate 851 89.7 490 86.5 .08 851 72.4 490 63.1 <.01e

Melanoma 429 49.2 245 52.2 .44 429 31.2 245 33.1 .62

Merkel Cell 420 91.4 266 88.7 .24 420 60.5 266 59.8 .85

GIST Stomach 433 94.2 249 84.3 <.01e 433 54.3 249 45.4 .03e

Ovary 427 94.9 245 89.4 <.01e 427 75.9 245 65.0 <.01e

Pharyngeal tonsil 225 56.9 112 54.5 .67 225 82.2 112 79.5 .54

Kidney Parenchyma

431 96.5 247 95.6 .53 431 96.5 247 96.0 .71

a=experience with Collaborative Staging system categories: “12 or more months of experience” versus “less than 12 month of experience”. b=number of distinct set of answers collected from participants and used for stage derivation. c=proportion correct derived stage (at the sub stage level). d=p‐value associated with Pearson chi‐square statistic (for the association between correct stage and CS version 2 experience). e=statistically significant


Pg. 26 of 32

Table 15. Comparison of Proportion Accurate Stage by Exposure to Organized CS v2 Training, for Certain Staging Systems and CS Schemas. CS v2 Schema Summary Stage 2000 AJCC Stage Group 7th edition

Some organized training

Self‐taught/on‐the‐job only Pd

Some organized training

Self‐taught/on‐the‐job only Pd

Nb % accuratec

Nb % accurate

c

Nb % accurate

c

Nb % accurate

c

Bladder 659 81.2 53 83.0 0.74 659 62.4 53 64.2 0.80

Breast 1,294 96.4 105 97.1 0.68 1294 82.3 105 80.0 0.55

Corpus Carcinoma

640 88.0 51 88.2 0.96 640 92.3 51 92.2 0.96

Brain 644 86.8 59 83.1 0.42 NA NA NA NA

Colon 1,263 79.8 118 70.3 0.02e 1263 89.6 118 84.8 0.11

Net Colon 632 57.0 51 54.9 0.78 632 64.9 51 60.8 0.56

Esophagus GE 640 88.6 54 72.2 <0.01e 640 63.8 54 57.4 0.35

Lung 1,272 90.9 102 86.3 0.13 1,272 89.2 102 85.3 0.22

Thyroid 306 100.0 26 100.0 1.00 306 69.6 26 69.2 0.97

Testis 629 88.7 51 90.2 0.75 629 29.7 51 33.3 0.59

Prostate 1,223 89.0 118 83.1 0.05 1,223 70.0 118 58.5 <0.01e

Melanoma 615 50.4 59 49.2 0.85 615 33.0 59 20.3 0.05e

Merkel Cell 635 90.1 51 88.2 0.59 635 60.9 51 51.0 0.16

GIST Stomach 623 91.2 59 84.8 0.11 623 52.0 59 40.7 0.10

Ovary 613 93.5 59 86.4 0.05e 613 72.6 59 64.4 0.18

Pharyngeal tonsil

307 56.0 30 56.7 0.95 307 82.4 30 70.0 0.10

Kidney Parenchyma

619 96.5 59 93.2 0.22 619 96.6 59 93.2 0.19

a=”Some organized training” category includes any respondent who attended, webinars or national or local in‐person training; “Self‐taught/on‐the‐job only” category includes respondents who did not receive training. b=number of distinct set of answers collected from participants and used for stage derivation. c=proportion correct derived stage (at the sub stage level). d=p‐value associated with Pearson chi‐square statistic (for the association between correct stage and attending organized training). e=statistically significant


Pg. 27 of 32

Figure 10. Risk of Inaccurate Staging by Education Achievement Category, adjusted for Certification Status, CS Schema, Abstracting Case Volume, and Experience with CS v2.

‐4% +20% ‐4%‐13%‐10%

SS2000 AJCC 7

Risk of Staging Errors by Education Achivement; reference Category = "Some College Education"

High school Some College Associate Degree

Bachelor Degree Gradute Education

Reference level

#= Reference level ‐ the average risk for inaccurate staging observed among study participants in the “some college” education category. Figure 11. Risk of Inaccurate Staging by Certification Status, Adjusted for Education Achievement, CS Schema, Abstracting Case Volume, and Experience with CSv2.

‐15% ‐12%‐39%** ‐32%**‐45%** ‐43%**

SS2000 AJCC 7

Risk of Staging Errors by Experience with CS v2; reference category = "No experience"

No experience Less than 6 mos. 6 mos. ‐ 1yr. 1yr. +

Reference level


Pg. 28 of 32

Figure 12. Risk of Inaccurate Staging by Experience with CSv2, Adjusted for Education Achievement, Certification Status, CS Schema, and Abstracting Case Volume.

#= Reference level ‐ the average risk for inaccurate

staging observed among study participants with no experience in CS v2. **= statistically significantly different than the reference category

Conclusion

This was the largest cancer registry reliability study ever done. The participants were experienced registrars and the database is robust.

The primary site of lung had the most inaccuracies. The most problematic variable was CS Extension. SEER Reliability Studies and CDC/PCR Audits over the past 5 years have shown the same results for the primary site and variable. The mean for CS Extension was 60.0% and the range was from 25.9% to 96.1% accuracy. Another area of concern is Lymph Nodes. The mean was 76.6% with a range of 46.0% to 100%. The variable, Behavior, had the highest accuracy with a mean of 99.7% and a range from 96.1% to 100%. Registrars with some organized training in CSv2 fared slightly better for Summary Stage 2000 than registrars that were self‐taught. The mean for those with training was 88.7% and the mean for those who were self‐taught was 84.8%.

Accuracy drops for AJCC Stage Group (7th Edition) and we see registrars with some CSv2 Training having a mean of 69.6% and registrars who have been self‐taught have a mean of 64.2% accuracy.

Higher education and experience with CSv2 both reduce the risk of staging errors. The transition from CSv1 to CSv2 has added complexity to staging that may not provide the level

of quality for staging data that is needed. Consideration should be given to reducing the complexity by collapsing codes where possible (where stage will not be affected).

The CSv2 Field Study Team hopes the data provided in this report assists the CSv2 Governance Committee to determine the best way to increase the quality of cancer stage data.

Ref. Ref.+16%+30%***

SS2000 AJCC 7

Risk of Staging Errors by Certification Status; reference category = "certified tumor registrar "

CTR (reference cat.) Non‐CTR

Reference level


Pg. 29 of 32

Tables, Protocol, Appendices will be on the CSv2 website at http://cancerstaging.org/cstage. Standard Setters involved in this reliability study have been given their specific data to perform additional reports as desired.

RECOMMENDATIONS Recommendations for Educational Topics

Retain study documentation to do a comparison Reliability Study on the same cases used in this study when CS is changed in the future for a true comparison for evaluation

Simplify how to code when documentation is not in the record o Prominent in coding for SSFs o Consider combining all the codes indicating the test was not done, unknown if

done; not applicable, etc. and putting them in one code.

Direct AJCC Lung Chapter Team for 8th Edition AJCC Staging Manual to provide more detail for lung staging

Develop short presentation on how to read the online manual and how to find answers o Education should not refer them to CAnswer Forum but to the CSv2 Manual page

that is appropriate o Reinforce that registrars MUST read or consult the CSv2 Manual for coding

Collapse “0” and “1” in CS Eval Code to “0” for “Clinical”

Clarify documentation in the CS Manual and provide further education on the following specific areas:

o Part I, Section I – Tumor Size – Coding when multiple sizes are available and clinical vs pathologic

o Part I, Section I & II – Lymph Nodes (includes CS LNs & related SSFs) –Coding clinical lymph nodes when there is no documentation

o Part I, Section II – SSFs – Clarification of “no histologic examination.” Breast – Lymph Nodes – Difference between code 250 and 600

o Breast – SSF #6‐Invasive/Insitu Components – Clarification and updating examples o Breast ‐‐ SSF #7‐Nottingham Score ‐‐ Clarification and examplesColon – Extension

and Lymph Nodes ‐‐ Anatomy of colon in regards to coding appropriate extension and lymph nodes

o Colon – SSF #7‐CRM – How to code when there is no mention of CRM or documentation of margins, NOS

o Lung – Extension – Understanding the anatomy of the lung o Lung – Mets at DX – Clarification Mets at DX codes, specifically pleural effusion o Lung – SSF 1: Separate Tumor Nodules – Examples of how to code o Lung – SSF 2: Pleura/Layer ‐‐ Examples of how to code o Prostate – Extension (Clinical) – Coding inapparent vs apparent o Prostate – SSF 3: Pathologic Extension – Emphasizing margin positive codes o Prostate – SSF 12, 13: Cores Positive and Examined – Examples of how to code o Bladder – Extension – How to interpret no muscle fragments available‐how to

code


Pg. 30 of 32

o Bladder – TS/Ext Eval – Emphasizing TURB is an eval 1 o Kidney – SSF 2: Vein Involvement – Examples of how to code o Kidney – SSF 3: Adrenal Gland Involvement – Examples of how to code o Kidney – SSF 4: Sarcomatoid Features – Examples of how to code o Melanoma Skin – Tumor Size and SSF 1: Breslow Depth’s – Examples of the

differences between the two and how to code o Melanoma Skin – Lymph Nodes – Examples of in‐transit mets and satellite nodules o Ovary – Extension – Examples of how to code peritoneal implants o Ovary – SSF 3: Residual tumor after cytoreduction surgery – Examples of how to

code o GIST Stomach – SSF 6: Mitotic Rate – Examples of how to code, specifically the

format o Corpus Carcinoma – Mets at DX – Definition of omental sampling and how it

relates to Mets at DX o Esophagus GE Junction – SSF 25: Schema Discriminator – Examples of how to

code APPRECIATION This 3 year project required the assistance of many cancer data professionals. We would like to acknowledge their work and dedication to the project.

CSv2 Field Study Team

Lynda Douglas (Co‐Leader) CDC/NPCR

Jennifer Ruhl (Co‐Leader) NCI SEER

Dave Annett IMS

Connie Bura CoC

Elaine Collins NPCR

Jean Cyr IMS

Brenda Edwards NCI SEER

Donna Gress AJCC

Jim Hofferkamp NAACCR

Gemma Lee Canada

Marty Madera CoC/AJCC

Zachary Myles CDC/NPCR

Jerri Linn Phillips CoC

Karen Pollitt AJCC

Dave Roney IMS

Shannon Vann NAACCR

CS Data Analysis Team

Serban Negoita (Leader) Westat

Jean Cyr IMS

Lynda Douglas CDC/NPCR

Stacey Fedewa ACS


Pg. 31 of 32

Missy Jamison NCI SEER

Carol Kosary NCI SEER Gemma Lee Canada

Zachary Myles CDC/NPCR

Jerri Linn Phillips CoC

Jennifer Ruhl NCISEER

Site Group 1

Jennifer Ruhl (Leader) NCI SEER

Cindi Dryer SEER

Terry Dawson NPCR

Bryan Palis CoC

Cyndy Russell Canada

Shannon Vann NAACCR

Site Group 2

Gemma Lee (Leader) Canada

Iris Chilton Canada

Kathy Malin CoC

Shawky Matta SEER

Nancy Santos SEER

Katheryne Vance NPCR

Site Group 3

Lynda Douglas (Leader) CDC/NPCR

Elaine Collins NPCR

Joann Janzen Canada

Erica McNamara CoC

Bobbi Matt SEER

Jeanne Whitlock SEER

Site Group 4

Jerri Linn Phillips (Leader) CoC

Sheena Batts SEER

Leah Driscoll SEER

Betty Gentry NPCR

Jim Hofferkamp NAACCR

Katharine Pearson Canada

Theola Rarick SEER

Tony Robbins ACS

Technical Expert Consultants

Lynn Ries Contractor

April Fritz Contractor

Annette Hurlbut Contractor (Transcription)

Mary Mesnard Westat


Pg. 32 of 32