Embed Size (px)
Citation preview
1
The ARDMS Item Writing
Handbook
2015
Version 4.0
2
ABOUT ARDMS
The American Registry for Diagnostic Medical Sonography® (ARDMS®), incorporated in 1975, is an independent, nonprofit organization that administers examinations and awards credentials in the areas of diagnostic medical sonography, diagnostic cardiac sonography, vascular technology, and vascular interpretation. ARDMS is widely considered the recognized global standard of excellence in sonography credentialing. ARDMS credentialing programs are accredited by International Standards Organization (ISO) and the National Commission for Certifying Agencies (NCCA).
ARDMS MISSION STATEMENT
ARDMS promotes quality care and patient safety through the certification and continuing competency of ultrasound professionals.
©2015 American Registry for Diagnostic Medical Sonography, Inc. All rights reserved. 2015 Version 4.0.0
3
ARDMS is the leading credentialing body for the sonography community. Our success is due to our volunteers who share the mission of quality care and patient safety. Since 1975, our exam questions and answers have been written by credentialed sonography professionals active in the field. The knowledge and experience of our dedicated volunteers ensure our exams are relevant and reflective of current practices.
Another part of ARDMS’ success has been our ability to revise the tests to reflect documented changes in the industry and improving testing techniques. For example, we no longer use “fill-in-the-blank” questions and offer only four, rather than five, possible answers to each question. To help both beginning and experienced item writers learn how to write quality items and adapt to item writing changes, we have produced a new The American Registry for Diagnostic Medical Sonography (ARDMS) Item Writing Handbook (the Handbook). This version of the Handbook is significantly different from previous editions, and we believe it provides you with better guidance on how to write and submit items for the various credentialing examinations. Some of the changes include:
1. An increased amount of white space to make it easier to read and take notes; 2. Call-out boxes highlighting the best practices for item writing; 3. A glossary that clearly defines item writing terminology; each term in the
glossary is underlined the first time it is used; 4. A tip sheet on the best practices in item writing and guidance on how to
avoid common errors; and 5. Clearly defined processes for submitting items and related images, digital
video clips and graphics.
We believe these updates to the Handbook will meet your needs and thus make it easier for you to write clinically pertinent quality items. If you have suggestions for further improvements to the Handbook, please contact [email protected]. Thank you for all your time and resources; you truly help us develop the best assessments possible for public safety. Sincerely, The ARDMS Testing Department
4
TABLE OF CONTENTS
ABOUT ARDMS .................................................................................................................... 2
ARDMS MISSION STATEMENT ............................................................................................ 2
A LETTER FROM THE ARDMS TESTING DEPARTMENT ........................................................ 3
PURPOSE OF THIS HANDBOOK ........................................................................................... 5
BACKGROUND INFORMATION ............................................................................................ 5
WHY SONOGRAPHY PROFESSIONALS TAKE ARDMS CREDENTIALING EXAMINATIONS ..... 5
COMPONENTS OF AN ARDMS CREDENTIALING EXAMINATION ........................................ 6
ANATOMY OF AN ITEM ....................................................................................................... 7
QUICK SUMMARY OF ITEM WRITING TIPS ......................................................................... 8
TIPS FOR DEVELOPING STRONG ITEM………………………………………………………………….......9-12
TIPS FOR WRITING QUALITY STEMS ............................................................................ 13-17
TIPS FOR WRITING QUALITY DISTRACTORS ................................................................. 18-19
APPENDIX A: ITEM DEVELOPMENT CHECKLIST ................................................................ 20
APPENDIX B: GUIDELINES FOR IMAGES AND DIGITAL VIDEO CLIPS ................................. 21
APPENDIX C: GLOSSARY .................................................................................................... 22
5
FYI: The Handbook uses examples to help illustrate the proper use of each tip. The correct answer in each example is marked with an asterisk (*).
PURPOSE OF THIS HANDBOOK
The American Registry for Diagnostic Medical Sonography (ARDMS) Item Writing Handbook Version 3.0 (the Handbook) provides both new and experienced item writers with guidance on how to prepare and submit items written for ARDMS credentialing examinations.
BACKGROUND INFORMATION
WHY SONOGRAPHY PROFESSIONALS TAKE ARDMS CREDENTIALING EXAMINATIONS
ARDMS credentials help patients, health-care professionals, and health-care employers identify the sonography professionals who meet or exceed the most rigorous sonography criteria and experience in the industry. Additionally, ARDMS credentials help sonography professionals:
Demonstrate their commitment to the ultrasound profession and its performance standards;
Remain current on new ultrasound trends and technology by requiring professionals to complete annual training classes; and
Establish their professional credentials by recognizing their individual, professional achievements.
6
COMPONENTS OF AN ARDMS CREDENTIALING EXAMINATION
ARDMS examinations consist of questions, better known as items. An item examines:
1. The basic information all entry-level professionals should know to be proficient in their field;
2. Professionals' ability to apply what they know in their work environment; and/or 3. The various operations or maneuvers all professionals use in their work
environment. Item content is derived from the Content Outline, a document developed by ARDMS staff and volunteer professionals summarizing the minimum amount of knowledge, skills and abilities sonography professionals are expected to master. Content outlines are available to item writers for all sonography credentialing examinations.
7
ANATOMY OF AN ITEM
The ARDMS uses a multiple-choice format, consisting of three parts, for its credentialing examinations:
1. The stem asks a question relating to a specific area of the content outline. 2. The key is the correct answer to the question asked in the stem. 3. Three distractors are plausible but incorrect answers to the question asked in the
stem. EXAMPLE: Axial resolution can be determined by evaluating which quality control test? (stem)
A. A flat interface (distractor) B. A round hole in a plastic block (distractor) C. Closely spaced parallel membranes (distractor) D. * Increasingly closely spaced fine wire (keyed correct response)
Best Practice: When developing items, always follow these guidelines:
Write all items in a question and answer format;
Use simple, clear language;
Focus on a single issue, fact, or concept per item;
Develop only four potential responses;
Concentrate on activities critical to everyday clinical practice, rather than trivial or experimental data;
Ensure that items test a person’s interpretation and problem solving skills rather than memory recall;
Use good grammar, punctuation, and spelling;
Do not use personal pronouns; and
Do not plagiarize.
Avoid abbreviations and acronyms.
8
QUICK SUMMARY OF ITEM WRITING TIPS
Click on a tip to advance to a more detailed explanation.
1. Keep items independent of one another.
2. Avoid verbatim phrasing from textbooks or journal articles in any part of an item.
3. Items should test only important and significant material.
4. Avoid giving unintended clues about the correct response.
5. Use correct grammar, punctuation and spelling, and do not use personal
pronouns.
6. Provide at least one reference for each item submitted.
7. Write the stem in the form of a question.
8. Focus on a single concept, fact or problem per item.
9. Use positive statements.
10. Place all common wording in the stem.
11. Avoid excessive use of measurements and numerical values.
12. Reference images in the stem.
13. Each item should have only one correct answer.
14. Use plausible distractors.
15. Delete specific determiners from the responses.
9
TIPS FOR DEVELOPING STRONG ITEMS
TIP 1. Keep items independent of one another. Do not write an item that requires information from a previous item in order to
be answered correctly. Obtaining the correct answer to one item should not be dependent upon obtaining the correct answer to another item. Additionally, avoid letting one item provide clues to the answer of other items.
EXAMPLE OF DEPENDENT ITEMS Which congenital heart defect features an overriding aortic root, ventricular septal defect (VSD), narrowed pulmonic artery, and right ventricular hypertrophy?
A. * Tetralogy of Fallot B. Ebstein anomaly C. Barlow syndrome D. Eisenmenger syndrome
Which Doppler finding is expected based on the information in the previous question?
A. Aortic stenosis B. * Pulmonic stenosis C. Mitral regurgitation D. Tricuspid regurgitation
EXAMPLE OF AN INDEPENDENT ITEM
Which finding is consistent with Tetralogy of Fallot?
A. Aortic stenosis B. * Pulmonic stenosis C. Mitral regurgitation D. Tricuspid regurgitation
10
TIP 3. Items should test only important and significant material.
TIP 2. Do not plagiarize. Copying exact language from textbooks and/or journal articles constitutes
plagiarism.
All items should be based on the correct examination’s Content Outline and should not be focused on trivial information, experimental techniques and/or unresolved industry debates. This allows ARDMS to test examinees' knowledge and analytical abilities.
11
TIP 4. Avoid giving unintended clues about the correct response.
Sophisticated test takers can detect inadvertent clues and select the right
response even if they otherwise would not know the answer. To avoid giving unintended clues:
Do not use similar words in both the stem and the responses
Develop responses that are approximately equal in length
Avoid making the key the longest or shortest response
WEAK ITEM EXAMPLE What are blood platelets’ functions?
A. Carrying oxygen to cells B. Carrying food to cells C. * Clotting of blood D. Fighting disease
STRONG ITEM EXAMPLE Which component of blood aids in clotting?
A. Serum B. * Platelets C. Monocytes D. Red blood cells
12
TIP 6. Provide at least one reference for each item submitted. All items written for ARDMS are required to have at least one reference. Items
without a reference will not be accepted.
Use sources that are commonly accepted in the subject area.
Recently-published articles and pending research are generally not appropriate references, since they are likely to contain material that is not yet commonly accepted or known.
TIP 5: Use correct grammar, punctuation and spelling, and do not use personal pronouns. Grammatical errors often help test takers identify the correct answer without
comprehending the subject matter.
Best practice: The Guide to Grammar and Writing at http://grammar.ccc.commnet.edu/grammar/ can help answer grammatical questions. The website www.dictionary.com is an excellent website to answer questions about spelling and word definitions.
Best Practice: When submitting an item reference, please include the title, author, publication date, and page(s) of the textbook or article used to develop the item.
13
TIPS FOR WRITING QUALITY STEMS
TIP 7. Write the stem in the form of a question. ARDMS uses only the “question and answer” format in its examinations. Do not
submit items that are formatted as incomplete sentences or “fill-in-the-blank.” WEAK ITEM EXAMPLE A small crater in a carotid artery plaque is called:
A. A thrombus B. * An ulceration C. A fibrous plaque D. A calcific plaque
STRONG ITEM EXAMPLE Which carotid plaque is associated with cerebral emboli?
A. Hyperechoic B. * Ulcerated C. Fibrous D. Calcific
14
TIP 8. Focus on a single concept, fact or problem per item.
A well-written stem allows the test taker to formulate a tentative answer even
before reading the potential responses. This is because only a single concept is being tested. Avoid adding unnecessary information to the stem, as it can unintentionally confuse test takers and increase reading time.
WEAK ITEM EXAMPLE Heart attacks are the number one cause of death in the United States, and coronary artery disease is preventable to some degree. Which is a risk factor for premature coronary artery disease?
A. Anemia B. Gall bladder disease C. Cancer D. * High cholesterol
STRONG ITEM EXAMPLE Which risk factor is associated with premature coronary artery disease?
A. Increased arterial elasticity B. Elevated high density lipoprotein (HDL) C. Decreased arterial pressure D. * Elevated low density lipoprotein (LDL)
15
TIP 9. Use positive statements. Positively written stems evaluate a test taker’s knowledge and actions while
limiting unnecessary confusion. WEAK ITEM EXAMPLE Which Doppler finding is NOT associated with severe aortic regurgitation (AI)?
A. Shortened AI pressure half-time (PHT)
B. Increased maximum aortic velocity
C. Diastolic flow reversal ascending aorta
D. Decreased left ventricular outflow tract (LVOT) / aortic velocity ratio
STRONG ITEM EXAMPLE Which echocardiographic finding is decreased in the presence of severe aortic regurgitation (AI)?
A. Left ventricular outflow tract (LVOT) / aortic velocity ratio
B. Proximal isovelocity surface area (PISA) radius
C. AI pressure half-time (PHT)
D. AI jet area
Best Practice: Avoid writing negative statements that use such words as “not” or “except.” If you are struggling with how to write a distractor, move on to another item and try rewording it later. Giving yourself a break from the question often allows your brain to find more appropriate wording.
16
TIP 10. Place all common wording in the stem. Avoid unnecessary repetition of material in the responses by including as much
information as possible in the stem. WEAK ITEM EXAMPLE What happens when a patient has severe proximal arterial occlusive disease?
A. The Doppler waveforms will not be distinguished from venous waveforms. B. The Doppler waveforms will demonstrate slight flow reversal in the diastolic
component. C. * The Doppler waveforms will appear markedly dampened, making
interpretation difficult for distal segments. D. The Doppler waveforms will eventually return to a relatively normal waveform
pattern once the disease process has stabilized.
STRONG ITEM EXAMPLE Which Doppler waveforms are consistent with severe proximal arterial occlusive disease?
A. Triphasic B. Diastolic flow reversal C. * Monophasic D. High resistance
17
TIP 11. Avoid excessive use of measurements and numerical values. Correct answers can only be determined when there are absolute standards. It
is best to avoid questions with measurements or numerical values, especially those that lack a published and professionally accepted absolute standard.
WEAK ITEM EXAMPLE What are the normal measurements of a uterus?
A. 6 cm long, 5 cm wide, and 4 cm thick B. * 8 cm long, 5 cm wide, and 4 cm thick C. 8 cm long, 6 cm wide, and 2 cm thick D. 10 cm long, 8 cm wide, and 5 cm thick
STRONG ITEM EXAMPLE A uterine measurement of 8 cm long, 5 cm wide, and 4cm thick, corresponds with which patient profile?
A. * Nulliparous B. Postpartum C. Pre-pubertal D. Post-menopausal
TIP 12. Reference images in the stem. Briefly refer to the existence of an image somewhere in the stem when
developing items involving an image. This will provide an indicator to the test taker to expect to see an image.
What does the sonographic finding in this image most likely represent?
Which finding is most consistent with the palpable mass identified in this image?
18
TIPS FOR WRITING QUALITY DISTRACTORS
TIP 13. Each item must have only one correct answer. Write items that are based on definitive and accepted facts and ultrasound
practices rather than opinion or debated issues. TIP 14. Use plausible distractors. Distractors should be credible and representative of misconceptions found in a
clinical setting. Most test takers will recognize unrealistic responses and be able to determine the correct answer with little thought or analysis of the concepts.
WEAK ITEM EXAMPLE What is it called when a patient can breathe comfortably only when sitting erect or standing?
A. Asthma B. * Orthopnea C. Pneumonia D. Emphysema
STRONG ITEM EXAMPLE Which condition is characterized by shortness of breath in the recumbent position?
A. Apnea B. Platypnea C. Dyspnea D. * Orthopnea
Best Practice: One useful strategy in generating distractors is to brainstorm all of the
potential misconceptions an entry-level professional may encounter. Use the most
realistic misconceptions as your distractors.
19
TIP 15. Delete specific determiners from the responses. Using specific determiners improves test takers’ chances for guessing the correct
answer without testing their analytical skills. WEAK ITEM EXAMPLE Which statement is correct regarding anencephaly?
A. It is associated with oligohydramnios. B. It is never associated with placenta previa. C. * It may be associated with polyhydramnios. D. It is usually followed with abruptio placenta.
STRONG ITEM EXAMPLE Which condition is associated with anencephaly?
A. * Polyhydramnios B. Abruptio placenta C. Decreased Maternal Serum Alpha-Fetoprotein (MSAFP) D. Two-vessel umbilical cord
Best practice: Avoid using words such as “all, always, never, frequently, often, commonly, rarely, usually, sometimes, typically, may, all of the above, and A, B, and C” when writing responses. These words encourage a test taker to quickly eliminate distractors without fully considering the options.
20
APPENDIX A: ITEM DEVELOPMENT CHECKLIST
Does the submitted item: CONTENT Present a single, definite concept in the stem? YES NO Have a single correct or best answer? YES NO Contain a key correct answer and THREE distractors? YES NO Test relevant information? (refer to Content Outline) YES NO Avoid trivial, overly technical or controversial issues? YES NO Avoid verbatim wording from texts or other references? YES NO Minimize references to measurements and numerical values? YES NO STYLE Place all common wording in the stem? YES NO Use positive statements only? YES NO Use correct grammar, punctuation and spelling? YES NO Read as a question? YES NO Refrain from giving unintended clues about the answer? YES NO Use plausible distractors? YES NO Limit variability of option length as a clue to the correct answer? YES NO Avoid specific determiners (e.g. “always, never, all of the above, etc.”? YES NO Stand independently from other items? YES NO Reference images in the stem? YES NO Spell out all words (no abbreviations or acronyms)? YES NO
21
APPENDIX B: GUIDELINES FOR IMAGES AND DIGITAL VIDEO CLIPS
All submitted material (questions, images, digital video clips, etc.) becomes the property of ARDMS. Please do not submit anything you have published, are planning to publish or that is used in any other venue.
1. Remove all identifying information (patient name, hospital name, etc.) to comply with relevant laws and regulations, including HIPAA. Please obtain appropriate permissions and authorizations
2. Please include only clips and frames that contain relevant information; however, color and Doppler along with 2D often completes the case study.
3. All images must be digital--no hardcopy images accepted. (Please no pictures from cellphones)
a. JPG (compressed) file format acceptable--use highest PPI (resolution) available
b. Should be at least 1024 x 768 pixels (width x height)
4. Color vs. black-and-white a. If image is color: use RGB (Red, Green, Blue). b. If image is black and white: use grayscale.
5. File naming
a. Use Item ID (provided by item authoring software) followed by the name of the diagnosis. You must save the item to generate the ID.
b. For example, "52341 ventricular septal defect"
22
APPENDIX C: GLOSSARY
Content outline – A document developed by ARDMS staff and volunteer professionals summarizing the minimum amount of information sonography professionals are expected to master before taking a certification examination. Distractor – Plausible but incorrect answer to the question asked in the stem. Examinee – A person taking an ARDMS certification examination. Item – A component of the examination designed to test a particular knowledge, skill, or
ability. It consists of both the stem and the responses (including the key and distractors). Item writer – An expert in a particular field who composes an item for an ARDMS examination. Key – The correct answer to the question asked in the stem. Responses – The multiple-choice alternatives following the stem, consisting of the keyed correct response plus three plausible distractors. Stem – Asks a question relating to a specific area of the content outline. Test-wise – Ability of an examinee who is skilled in detecting inadvertent clues to the right answer for an item, even when they otherwise would not know the answer.
