Course learning objectives Course learning opportunities Learning assessments for students

Review of Y2 SBM/CT and MS CoursePreClinical Subcommittee

November, 2013Chris Rees (Geisel 2), Steve Benson MD, Dave Nierenberg

MD

1. Course learning objectives

2. Course learning opportunities

3. Learning assessments for students4. Measures of overall quality for the course5. Specific suggestions for improving course

1. 2012-2013 Course Learning Objectives # Objective Maps to

1 Apply current knowledge in MS Medicine to Dx and Rx decisions 1a

2 Apply current clinical and translational science to Dx and Rx questions 1b,c,d

3 Apply knowledge of prevention, risk factor modification, substance abuse, pain management, medical ethics, and medical-legal issues to clinical problem solving

1e

4 Apply knowledge of impact of social, economic, cultural and personal factors on health to clinical problems in MS medicine

1f

5 Establish mutually respectful student-patient and student-family interactions with diverse patients

2a,c,3a,b

6 Interview patients skillfully, using a MS-focused history 2b,2c

7 Examine patients with MS conditions skillfully, with attention to cleanliness, infection control, and patient comfort

2c

8 Generate an appropriate differential Dx for patients with MS disorders 2d,e

9 Explain the indications, complications, and performance of diagnostic tests (e.g. CT, MRI, EMG, NC, tissue biopsy)

2f,g

10 Identify common abnormal findings on plain radiographs, CT scans, and MRI scans

2h

11 Communicate effectively with patients and families especially when special barriers to communication exist (eg. Cultural, socioeconomic, linguistic, etc)

3c

2012-2013 ILIOS Course Objectives (cont’d)# Objective Maps to12 Assist patients in understanding their treatment options and prognosis 3d

13 Communicate collegially and effectively, verbally and in writing, with physician colleagues and other members of the HC team

3e,f,g,h

14 Behave respectfully and responsibly towards patients, families, colleagues, and other members of the HC team, and remain aware of different concerns, opinions, and cultural perspectives

4a,e

15 Meet professional responsibilities fully 4b

16 Adhere to high ethical and moral standards,, accept constructive criticism, and respect patient confidentiality

4d,f,g

17 Take responsibility for one’s own medical education, and develop habits of mindfulness and reflection

4h,j

18 Describe barriers to access basic health services, and its effect on vulnerable populations

4k

19 Contribute constructive feedback during peer review 4l

20 Identify and critically evaluate evidence-based and cost-conscious strategies for patients with MS diseases, and apply these approaches to patient care

5a,b,e,f

21 Assess the effect of social environment on clinical care and patient outcomes

5c

22 Identify appropriate resources to support patient care, and collaborate effectively with all members of the inter-professional team

6a,b

1a. Learning Objectives (Dave) In the course syllabus 2013, the course objectives are listed

as follows: To provide a thorough and detailed introduction to

rheumatologic conditions and musculoskeletal medicine To know the important historical features, physical

findings, laboratory and radiographic findings that help differentiate one connective tissue disease from another

To develop a structured approach to the patient with musculoskeletal symptoms in order to construct an appropriate differential diagnosis and develop rational diagnostic and therapeutic plans.

Students are not provided with the ILIOS course objectives Important to have one shared set of core learning objectives,

presented right at the beginning of the course

Continued (Dave) From the lecture topics, and also the categories on the final exam, we can see the

core topics of the course include: Overview of course, and the approach to the patient Elements of the joint and MS exam Osteoarthritis Rheumatoid arthritis Pediatric conditions and JRA Lyme disease Other types of arthritis: seronegative, crystal, infectious Muscle conditions, myositis and myofascial pain Vasculitis, SLE, and scleroderma Lab testing Soft tissue problems and injuries Bone diseases, fractures, and tumors DJD and other Diseases of the spine Congenital and developmental problems Imaging the MS and CT systems Common genetic conditions

1. Topics in national textbook(Cecil Essentials of Medicine, 2013)

Approach to patient with rheumatic disease (Yes)

Rheumatoid arthritis (Yes) Spondyloarthropathies (Yes) SLE (Yes) Antiphospholipid antibody

syndrome (yes) Systemic sclerosis (Yes) and

Sjogren’s syndrome Idiopathic inflammatory

myopathies (Yes)

Sjogren syndrome (no in 2013, yes in 2014)

Systemic vasculitis (Yes) Crystal arthropathies (Yes) Osteoarthritis (Yes) Non-articular soft tissue

disorders (Yes) Rheumatic manifestations of

systemic disorders (Yes)

1f. Learning objectives addressing additional specific LCME topics (with Nicole) Health of populations:

Senior epidemic of OA, and how obesity contributes to the clinical progression

Chronic pain syndromes, e.g. fibromyalgia Basic and ethical principles of clinical and translational

research: How the intricate process of basic and translational immunology

research is changing the way we approach pts with RA (William Rigby) Gender and cultural biases in students themselves:

Not at this time Instruction in medical ethics and human values

In 2014 will be their first Hearts and Minds session about a patient with fibromyalgia (chronic pain syndrome) and her long-term use of opioids for pain management

1g. Overlap/ILIOS word search: “Vasculitis” Does not appear in any course objectives Appears in one session title (SBM/MS) Appears in 3 sessions in 3 different courses

1g. Overlap/ILIOS word search: “Rheumatoid Arthritis” Does not appear in any course objectives Appears in one session title (SBM/MS) Appears in 5 sessions in 3 different courses

1g. Overlap/ILIOS word search: “uric acid”(note: there are other “hits” when “gout” is searched)

Does not appear in any course objectives Does not appear in any session title Appears in 4 sessions in 4 different courses

1. Learning Objectives: Summary

Issue Proposed FixLearning objectives in ILIOS are vagueLearning objectives in ILIOS differ from those posted in the courseGroup learning objectives by major competency domain, and map to specific competency tasksCurrent course objectives in ILIOS do not contain the key words of the major diseases and conditionsMake sure that exam topics track nicely to new learning objectivesList learning objectives for each session at the beginning of each session

2. Course Learning Opportunities 2012-13 (Dave)

Total hours: 39 h Traditional lectures = 29 h (74% of total hours)

Large group discussion/review = 0 h Lab (classical) = 0 Small groups/conferences = 6 PBL groups = 3 Direct patient contact or PE session = 1 (2 groups) Panel discussion (Hearts and Minds) = 0 Other formats = 0

2. Learning Opportunities Summary

Issue Proposed FixOver-reliance on standard lectures (29/39=74%)Not enough engaged large group sessions (0)Only 6 h of conference groupsDidn’t have Hearts and Minds panel discussion in 2013Current 1 h PE demonstration clinic is excellent

3. Content of Final exam (current topics 2013)Topic # Topic #Acute soft tissue 6 Myositis 3Bone pathology 3 Osteoarthritis 7Bone tumors 1 Pediatric topics 3Development 1 Physical dx 4DJD of spine 2 Imaging 1Fibromyalgia 4 Rheumatoid arthritis 11Fractures 1 Scleroderma 5Genetics 1 Seronegative arthritis 3Gout 7 SLE 10Infectious arthritis 4 Testing 2JRA 3 Vasculitis 5Lyme disease 5 Pharmacology 6

3. 2012-2013 Exam items sorted by current course learning objectives (Steve)

# Objective Maps to

# items

1 Apply current knowledge in MS Medicine to Dx and Rx decisions 1a 70

2 Apply current clinical and translational science to Dx and Rx questions

1b,c,d 1

3 Apply knowledge of prevention, risk factor modification, substance abuse, pain management, medical ethics, and medical-legal issues to clinical problem solving

1e 2

4 Apply knowledge of impact of social, economic, cultural and personal factors on health to clinical problems in MS medicine

1f 0

5 Establish mutually respectful student-patient and student-family interactions with diverse patients

2a,c,3a,b

0

6 Interview patients skillfully, using a MS-focused history 2b,2c 0

7 Examine patients with MS conditions skillfully, with attention to cleanliness, infection control, and patient comfort

2c 7 PE findings

8 Generate an appropriate differential Dx for patients with MS disorders

2d,e 1

9 Explain the indications, complications, and performance of diagnostic tests (e.g. CT, MRI, EMG, NC, tissue biopsy)

2f,g 2

10 Identify common abnormal findings on plain radiographs, CT scans, and MRI scans

2h 9

11 Communicate effectively with patients and families especially when special barriers to communication exist (eg. Cultural, socioeconomic, linguistic, etc)

3c 0

2012-2013 Exam items (cont’d)# Objective Maps to # items

12 Assist patients in understanding their treatment options and prognosis

3d 0

13 Communicate collegially and effectively, verbally and in writing, with physician colleagues and other members of the HC team

3e,f,g,h 0

14 Behave respectfully and responsibly towards patients, families, colleagues, and other members of the HC team, and remain aware of different concerns, opinions, and cultural perspectives

4a,e 0

15 Meet professional responsibilities fully 4b 0

16 Adhere to high ethical and moral standards,, accept constructive criticism, and respect patient confidentiality

4d,f,g 0

17 Take responsibility for one’s own medical education, and develop habits of mindfulness and reflection

4h,j 0

18 Describe barriers to access basic health services, and its effect on vulnerable populations

4k 0

19 Contribute constructive feedback during peer review 4l 0

20 Identify and critically evaluate evidence-based and cost-conscious strategies for patients with MS diseases, and apply these approaches to patient care

5a,b,e,f 0

21 Assess the effect of social environment on clinical care and patient outcomes

5c 0

22 Identify appropriate resources to support patient care, and collaborate effectively with all members of the inter-professional team

6a,b 0

3. Exam content (MS final exam)Question style # Factual only, test recognition/memorization

Important concepts, avoid test fatigue 15 Clinical vignette with reasoning

Excellent, well written, pertinent 59

Application of knowledge Interpret findings, data 7 Photos of PE findings 8

Negative stem 2 (only!) Multiple T/F format 1 (only!)

Pharmacology questions 64 were simple recall, 2 clinical vignettes

3. Evaluating Students, SummaryIssue Proposed FixLearning issues need to be granular enough that major exam areas map well to them

Learning objectives not assessed on written final exam must be assessed in some other way

Conference leaders may not know students well enough to assess their competency in multiple areasConference size may be too large to enable faculty to get to know each student

Written final exam made good use of vignettes, fewer factoid questions, very few negative stem

4. Performance on USMLE Step 1 over last 3 years

4. Feedback about course from March 2012 AAMC

GQHow well did each of the following sciences

basic to medicine prepare you for clinical clerkships and electives?

All Year 1 disciplines (n=9): mean = 3.1All Year 2 disciplines (n=4): mean = 3.4 (3.3)Renal course was not broken out

4. Feedback about course:Student survey scores (Dave)

Results from recent student course reviews

Data from 2012-2013 evaluations:Strongest areas:

• “Overall usefulness of attending laboratory sessions or simulated laboratory sessions.” 4.22

• “How well this course provided me with a useful and appropriate introduction to this field of discipline.” 4.05

• “Overall quality of the entire course” 4.00

Among the questions asked on the course review pertaining to the “usefulness” of course elements, none scored below at 3.48.

All faculty were rated at a 3.16 or higher, with 16 of 20 lecturers receiving a score of 3.70 or higher.


Representative comments (strengths): Many students commented on the fairness and

congruence of the final exam. “The clinic visit was fantastic. This should be standard for

all courses.” “The best part of the course was the conferences.” The majority of faculty received overwhelmingly positive

scores from the students.


Representative comments (weaknesses): Several students commented that the clinical experience

would be more useful later in the term. (done for 2014) “Perhaps one or two more conference groups would be

helpful.” (done for 2014) “The organization of the course could be a little better.

The arthritis lectures could be grouped together and the bone lectures could be grouped together.” (done for 2014)

Some students commented that the required paper felt like “busywork.”(replaced in 2014 by off-line quizzes)

4. Global measures of qualityIssue Proposed FixShift clinical experience (PDX session) later into termHaving one or two more conference group meetings would be helpful (highlight of course)Improve course organization by better grouping of topics (e.g. arthritis material, bone material)Required written paper (analyzing a journal article) felt like “busywork” to a few students

PLANS FOR ADDRESSING CONCERNS AND ISSUES

FOR NEXT CYCLE, TERM 4, 2014

1. Learning Objectives: SummaryIssue Proposed FixLearning objectives in ILIOS are vague

Rewrite new set of course learning objectives

Learning objectives in ILIOS differ from those posted in the course

Post the new ILIOS objectives in syllabus on first day of course

Group learning objectives by major competency domain, and map to specific competency tasks

Done in new version

Have enough detail in new course learning objectives to permit ILIOS word searches

Done in new objectives

Make sure that exam topics track to new learning objectives

Done in new objectives

List learning objectives for each session at the beginning of each session

Will do next cycle

DRAFT New Course Learning Objectives for 2013-14

Each student should demonstrate an appropriate level of understanding of the pathophysiology of the musculoskeletal/connective tissues system, such that the student is prepared to recognize, diagnose, and describe effective treatment options for the most common and severe diseases and disorders of this organ/system that may be encountered during the clinical clerkships or in clinical practice.

Possible Course Learning Objectives 2013-14# Objective Domain Assessment

1 Describe the pathophysiology, diagnosis, and treatment options for osteoarthritis 1abcde Conference, PBL, exam

2 Describe the pathophysiology, diagnosis, and treatment options for rheumatoid arthritis (including juvenile rheumatoid arthritis)

1abcde Conference, PBL, exam

3 Describe the pathophysiology, diagnosis, and treatment options for less common types of arthritis (e.g. sero-negative, psoriatic, etc)

1abcd Conference, PBL, exam

4 Describe the pathophysiology, diagnosis, and treatment options for diseases of the lumbar and cervical spine (including DJD, ankylosing spondylitis, disc disease, low back pain,etc)


5 Describe the pathophysiology, diagnosis, and treatment options for complications of uric acid and other crystals (including gout, pseudogout)


6 Describe the pathophysiology, diagnosis, and treatment options for common infections of the MS system (e.g. Lyme disease, septic arthritis)


7 Describe the pathophysiology, diagnosis, and treatment options for diseases of skeletal muscle (e.g. myositis, polymyositis, etc.)


8 Describe the pathophysiology, diagnosis, and treatment options for connective tissue diseases (eg. systemic lupus erythematosus, scleroderma, mixed CT disease, Sjogren’s syndrome)


9 Describe the pathophysiology, diagnosis and treatment options for the various types of vasculitis (eg. Polymyalgia rheumatica, giant cell arteritis, granulomatosis with polyangiitis, polyarteritis)


10 Describe the pathophysiology, diagnosis, and treatment options for bone and soft tissue diseases (e.g. tumors, fractures, sprains)


11 Describe the pathophysiology, diagnosis, and treatment options for common pediatric and developmental diseases (e.g. juvenile rheumatoid arthritis, genetic conditions)


12 Describe the advantages and disadvantages of major diagnostic testing modalities for the MS system (e.g. radiography, CT, MRI, ultrasound, serologic testing, etc.)

2fgh Conference, PBL, exam

Possible Course Learning Objectives 2013-14# Objective Domain Assessment

13 Describe the most common symptoms of rheumatic and musculoskeletal diseases, and use that knowledge to help develop a differential diagnosis

2ab Conferences, PBL

14 Explain how patients with MS/CT disorders can be skillfully and respectfully examined, with appropriate attention to cleanliness and privacy

2ac Rheumatology clinic

15 Explain the indications, complications, and limitations of common diagnostic modalities including imaging studies, EMG, NCS, and tissue biopsies

2fg Exam

16 Correctly identify common abnormalities seen on plain radiographs, CT scans, and MRI’s of the MS system

2gh Conference, PBL, exam

17 Communicate effectively, verbally and in writing, with colleagues and physicians 3efg Conference, PBL

18 Demonstrate how to assist patients in understanding their treatment options, and prognosis of their diseases

3abcd Conference, PBL

19 Meet professional responsibilities (meetings, assignments, etc) in a timely and professional manner

4bdfh Conference, PBL

20 Demonstrate the ability to take responsibility for one’s own medical education 4h Conference, PBL

21 Describe how barriers to access for care of MS/CT problems adversely effects the health of vulnerable populations

4k Conference, PBL

22 Demonstrate the ability to identify and evaluate information about evidence-based and cost-conscious strategies in managing MS/CT disorders

5abce Conference, PBL, exam

23 Identify appropriate resources to help support patients with MS/CT disorders, and describe the rolls of all members of the inter-professional HC team

6abef Conference, PBL

24 Describe the appropriate role of rehabilitation (physiatry) and related services (OT, PT, etc) in the care of patients with MS/CT diseases

6abce Conference, PBL

2. Learning Opportunities SummaryIssue Proposed FixOver-reliance on standard lectures (29/39=74%)

In 2014, will reduce to 23/39 (59%)

Not enough engaged large group sessions

Will increase from 0 to 3 in 2014

Only 6 h of conference groups Will increase to 8 h in 2014Don’t have Hearts and Minds panel discussion in 2013

Will introduce new 1h session in 2014

Current 1 h demonstration clinic is excellent

This will be continued in 2014

3. Evaluating Students SummaryIssue Proposed FixLearning issues need to be granular enough that major exam areas map well to them

Fixed with new course learning objectives

Learning objectives not assessed on written final exam must be assessed in some other way

Assigned in new course learning objectives to observation of students in conference, PBL groups

Conference leaders may not know students well enough to assess their competency in multiple areas

Increase hours spent with same conference leader

Conference size may be too large to enable faculty to get to know each student

Increase number of faculty-led conference groups

Written final exam made good use of vignettes, fewer factoid questions, very few negative stem

Continue to develop new items for final exam

4. Feedback from StudentsIssue Proposed FixShift clinical experience (PDX session) later into term

Fixed for 2014

Having one or two more conference group meetings would be helpful (highlight of course)

Fixed for 2014

Improve course organization by better grouping of topics (e.g. arthritis material, bone material)

Done for 2014

Required written paper (analyzing a journal article) felt like “busywork” to a few students

Re-evaluate goals of this exercise: consider asking for a shorter paper, earlier in the course, that will help students develop the skills to translate translational research findings into language patients can understand

Documents

Course learning objectives Course learning opportunities Learning assessments for students