SJSU Annual Program Assessment Form Academic Year 2015‐2016

Instructions 1. Complete the attached form and submit it as an email attachment to Graduate and

Undergraduate Programs (academicassessment@sjsu.edu) on or before June 1, 2016.

2. Please copy your college’s Associate Dean and Assessment Facilitator on the email submission.

Assessment Facilitators are also available to provide support - please feel free to contact them with any questions or concerns.

3. Completed forms will be posted on your Program Records webpage.

Please note that this form has been updated since last year. We have made several minor changes that we believe will streamline the reporting process and increase focus on the implementation of changes based on assessment results (“Closing the Loop”). The program data elements (graduation rates, headcounts, SFR, etc.) have been dropped from this annual assessment report. This data is still available through the Institutional Effectiveness and Analytics (IEA) website and we encourage programs to examine this data on a regular basis. However, this information will only be required to be reported as part of the Program Planning process. This report is organized into three sections designed to organize your annual assessment efforts and to inform your department’s Program Planning. Here is the rationale behind each section.

Part A – The Big Picture ● This section will likely only need to be prepared once at the beginning of your assessment cycle,

although it should be reviewed each year and updated as necessary. This information should be included in each annual report, even if it has not changed.

● This section lists your Program Learning Outcomes (PLOs) and, more importantly, how they connect with your curriculum within the program and the University Learning Goals (ULGs).

● Finally, this section presents your assessment plan for the current planning cycle in the form of a multi-year schedule (usually 5 years, updated as part of Program Planning). This schedule should indicate which PLO(s) will be assessed each year, as well as your plans for implementing changes based on assessment results, and re-assessment after changes have been given time to take effect.

Part B – What We Did This Year ● This section details your assessment efforts over the last year (AY 2015-16). ● Which PLO(s) were assessed, how was the data collected, and what do the data tell you with

regard to student achievement on this PLO? What do you plan to do, if anything, to improve future achievement levels (i.e., “close the loop”)?

Part C – Keeping Track of the Changes (“Closing the loop”) ● This section is meant to keep a running record of your efforts to improve your students’

outcomes. This table should grow throughout your assessment cycle and will be an important part of your next Program Plan.

● Create a new row in the table each time you propose a change as a result of your assessment efforts. Then be sure to keep track of your change efforts in subsequent years.

SJSU Annual Program Assessment Form Academic Year 2015‐2016

Department: Biomedical, Chemical and Materials Engineering

Program: B.S. Biomedical Engineering

College: Engineering

Program Website: https://bcme.sjsu.edu

Link to Program Learning Outcomes (PLOs) on program website: https://bcme.sjsu.edu/node/688

Program Accreditation (if any): Expected in August 2016

Contact Person and Email: Alessandro Bellofiore, alessandro.bellofiore@sjsu.edu

Date of Report: 06.01.2016

Part A

1. List of Program Learning Outcomes (PLOs)

The Program Learning Outcomes, which are consistent with the Accreditation Board for Engineering and Technology (ABET) disciplinary standards and requirements, and the Program Evaluation Components (PEC) for each PLO, are listed below. The PECs for each PLO were identified and developed to ensure that the assessment for each PLO would be relevant to biomedical engineering.

The faculty as a group decide and formulate, through several discussions, the specific PECs that are relevant to biomedical engineering, and the level of mastery required, keeping in mind the requirements that can be expected to be placed when our students graduate and enter the professional workforce. The PECs are reviewed annually and modifications made if the faculty as a group think that is warranted.

1. Ability to apply knowledge of mathematics, science and engineering

1.1 Identify the basic structural and functional principles of human organ systems including repair systems 1.2 Apply conservation laws to biological and medical systems to solve biomedical engineering

problems. 1.3 Apply engineering fundamentals and scientific reasoning to model and predict responses at

biological interfaces 2. Ability to design/conduct experiments and analyze/interpret data

2.1 Design and analyze appropriate experiments to measure or optimize specific engineering properties, incorporating statistical procedures

2.2 Analyze and interpret results of specific and mandatory FDA testing 2.3 Solve closed and open-ended biomedical engineering problems using experimental

methodologies

3. Ability to design a system, component, or process to meet desired needs within realistic

constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability 3.1 Demonstrate an understanding of the use of different and biomedical devices in terms of

safety and efficacy 3.2 Demonstrate knowledge of the constraints around process/product design based on FDA

regulations 4. Ability to function on multi-disciplinary teams

4.1 Function effectively as both team leader and team member in accomplishing engineering team projects

5. Ability to identify, formulate and solve engineering problems

5.1 Troubleshoot a biomedical system by dividing the system into subcomponents and narrow the failure to single subsystem or an interaction

5.2 Simulate the problem by using mathematical modeling tools 5.3 Evaluate the constraints in a biomedical engineering problem and develop solutions

6. Understanding of professional and ethical responsibility

6.1 Formulate and address ethical issues which arise in solving engineering problems and in the workplace

7. Ability to communicate effectively

7.1 Communicate effectively in informal team settings and through formal and informal presentations, in written and oral formats

8. Understand the impact of engineering solutions in a global/societal context

8.1 Understands consequences of global/societal context of biomedical engineering issues and policies.

9. Recognition of the need for and an ability to engage in life-long learning

9.1 Conduct a thorough information search, be resourceful in uncovering information, and critically evaluate information.

10. Knowledge of contemporary issues

10.1 Compare and evaluate current and emerging biomedical engineering technologies

11. Ability to use the techniques, skills and modern tools necessary for engineering practice 11.1 Apply appropriate software, modern tools, and techniques for design and analysis of

biomedical systems

2. Map of PLOs to University Learning Goals (ULGs)

Mapping of the PLOs to the University Learning Goals was initially done by the Biomedical Engineering Program Director, and then reviewed and discussed by the biomedical engineering faculty and adjustments made. Table 1 contains the mapping that evolved as a result of this collaborative review process.

The five University Learning Goals are listed below: 1. ULG #1 ‐ Specialized Knowledge: Depth of knowledge required for a degree, as identified by its program learning outcomes 2. ULG #2 ‐ Broad Integrative Knowledge: Mastery of each step of an investigative, creative, or practical project. Understanding of the implications of results or findings from a particular work in societal context 3. ULG #3 ‐ Intellectual Skills: Fluency in the use of specific theories, tools, technology, and graphical representation. Skills and abilities necessary for life‐long learning: critical and creative thinking effective communication, conscientious information gathering and processing, mastery of quantitative methodologies, and the ability to engage effectively in collaborative activities 4. ULG #4 ‐ Applied Knowledge: Ability to integrate theory, practice, and problem‐solving to address practical issues. Ability to apply their knowledge and skills to new settings or in addressing complex problems. The ability to work productively as individuals and in groups 5. ULG #5 ‐ Social and Global Responsibilities: Ability to act intentionally and ethically to address a global or local problem in an informed manner with a multicultural and historical perspective and a clear understanding of societal and civic responsibilities. Diverse and global perspectives through engagement with the multidimensional SJSU community

Table 1: Mapping of Biomedical Engineering Program Learning Outcomes to University Learning Goals

BS Biomedical Engineering Program Learning Outcomes (PLOs)

University Learning Goals

1 2 3 4 5

(a) Ability to apply knowledge of mathematics, science and engineering X X

(b) Ability to design/conduct experiments and analyze/interpret data X X

(c) Ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability

X X X

(d) Ability to function on multi-disciplinary teams X X X

(e) Ability to identify, formulate and solve engineering problems X X X

(f) Understanding of professional and ethical responsibility X

(g) Ability to communicate effectively X X

(h) Understand the impact of engineering solutions in a global/societal context

X X

(i) Recognition of the need for and an ability to engage in life-long learning X X X

(j) Knowledge of contemporary issues X X

(k) Ability to use the techniques, skills and modern tools necessary for engineering practice

X X

3. Alignment – Matrix of PLOs to Courses

The courses in which each Program Evaluation Component is assessed, the semester during which it was assessed, and the level at which it was assessed, during AY 2014-2015, is shown in Table 2. The assessment schedule for AY 2016-2017 is expected to be similar to the AY 2014-2015 assessment

schedule. The specific schedule and the Program Evaluation Component to be assessed will be finalized during the Biomedical Engineering Program meeting during Summer 2016.

Table 2: Matrix showing specific courses, semesters, and levels at which Program Evaluation Components are assessed.*

F14 S15 S15 S15 F14 F14 F/S 14-15

Program Evaluation Components BME 115

BME 117

BME 173

BME 174

BME 177

ChE 162

BME 198AB

1.1 Identify the basic structural and functional principles of human organ systems including repair systems 1 2

[2]

1.2 Apply conservation laws to biological and medical systems to solve biomedical engineering problems.

[2] 2

3

1.3 Apply engineering fundamentals and scientific reasoning to model and predict responses at biological interfaces

1 2

[2]

3

2.1 Design and analyze appropriate experiments to measure or optimize specific engineering properties , incorporating statistical procedures

2

2

2.2 Analyze and interpret results of specific and mandatory FDA testing

2 [3]

3

2.3 Solve closed and open-ended biomedical engineering problems using experimental methodologies

2

[3]

3.1 Demonstrate an understanding of the use of different and biomedical devices in terms of safety and efficacy

[2]

2

3.2 Demonstrate a knowledge of the constraints around process/product design based on FDA regulations 1

2 [2]

4.1 Function effectively as both team leader and team member in accomplishing engineering team projects 1 1

[2]

5.1 Troubleshoot a biomedical system by dividing the system into subcomponents and narrow the failure to single subsystem or an interaction

[1] 2

5.2 Simulate the problem by using mathematical modeling tools

1 [1]

2

6.1 Formulate and address ethical issues which arise in solving engineering problems and in the workplace 1

[3] 3

7.1 Communicate effectively in informal team settings and through formal and informal presentations, in written and oral formats

1 1 2 2

[3]

8.1 Understands consequences of global/societal context of biomedical engineering issues and policies.

2 [2]

3

9.1 Conduct a thorough information search, be resourceful in uncovering information, and critically evaluate information.

1

2 2

[3]

10.1 Compare and evaluate current and emerging biomedical engineering technologies 1

2 2

[2]

11.1 Apply appropriate software, modern tools, and techniques for design and analysis of biomedical systems

1 1 1 1

[2]

* Relationship to Bloom’s Taxonomy: Level 1 in this table corresponds to Bloom’s Levels I & II, Level 2 corresponds to Bloom’s Levels III & IV, and Level 3 corresponds to Bloom’s Levels V & VI.

4. Planning – Assessment Schedule

The assessment schedule for the next eight semesters is shown in Table 3. The schedule includes a plan for PLO assessment (A), implementation of changes as a result of the assessment (I), and PLO reassessment to gauge the impact of the change (R).

Table 3: Matrix showing specific courses, semesters, and levels at which Program Evaluation Components are assessed.*

F16 S17 F17 S18 F18 S19 F19 S20

PLO 1 A I R A I R PLO 2 A I R A I R PLO 3 A I R A I R PLO 4 A I R A I R PLO 5 A I R A I R

5. Student Experience The Program Learning Objectives and ABET Program Educational Objectives are available in the university catalog and posted on the Biomedical Engineering Program website at the following:

- Program Learning Objectives: https://bcme.sjsu.edu/node/688 - ABET Program Educational Objectives: https://bcme.sjsu.edu/BS-BME-Program-Objectives

Additionally, specific Learning Outcomes are included in every course syllabus. These outcomes generally reflect Program Learning Objectives. The Program Learning Objectives are consistent with the requirements of ABET. If we are to value ABET accreditation, there is not much room for making changes to the PLOs. The Program Evaluation Components (PECs), however, were developed by the program faculty. Consideration of student feedback in the formulation of the PECs, while being a desired element, can be very difficult to implement as the students, especially undergraduates, do not have sufficient knowledge of the field, which is extremely broad, to be able to provide meaningful input. Input from the program's Industry Advisory Council was sought and included in the formulation of the PECs. In addition, input from the program's Industry Advisory Council as well as Program alumni is sought and considered in any changes made to the PECs.

Part B

6. Assessment Data and Results

Assessment of all the PECs was completed in the AY 2014-15 to fulfill the requirements for the accreditation visit by ABET in Fall 2015.

All full-time and part-time program faculty participated in the assessment process. The Program Evaluation Criteria (PECs) associated with the Student Outcomes were assessed in the pertinent courses by the instructor who taught that particular course. Attainment of the PECs is evaluated based on a combination of homework questions, quizzes, examination questions, in-class oral presentations, poster presentations, and term papers/projects. The specific combination of evaluation instruments used varies somewhat from class to class, depending on the particular class. The goal is to have more than one direct assessment method for each of the PECs outcomes. At the conclusion of each semester each faculty member completes a report for the class (or classes) that s/he was teaching. This is completed during the winter break and at the beginning of the summer for classes that were taught during the fall and spring semesters, respectively. The level at which a Student Outcome is to be assessed, and the course in which it is to be assed, is decided by the BME faculty jointly. The actual assessment in a particular class is then executed by the faculty member teaching that class. Faculty who assess their outcomes identify what the minimum acceptable level of performance for each of the outcomes should be. This is different for each instructor and for each outcome. For some of the outcomes, a specific rubric that describes what mark (score) is given, based on the student’s performance with respect to that outcome, is assembled. It is desired that 100% of the students meet at least the minimum acceptable level of performance on each of the criteria. While this does not always happen in reality, it enables the faculty to identify and implement different pedagogical strategies with the aim of improving their teaching methods and the students’ performance.

7. Analysis Analysis of the assessments of the student outcomes and the level of attainment for the 2014-2015 academic year is summarized in the following. Assessment data are reported in Appendix A. A very brief description of the instruments used for the assessment is also included. With respect to BME 115 (Foundations of Biomedical Engineering), all students attained the minimum acceptable level of attainment of 50% for PEC 1.2. For PECs 2.1 and 2.2, 90% of the students attained 75% or higher. For BME 117 (Biotransport Phenomena) for PECs 1.2 and 1.3, 81 % of the students attained 75% or higher, with the acceptable level being 70%. For PEC 5.2, more than 80% of the students attained the acceptable level of performance. In BME 173 (Clinical Trials in Biomedical Engineering) PECs 3.1 and 6.1 were assessed, with the minimum acceptable attainment level being 70%. Of the 29 students enrolled in the class, 25 students attained the required level while four did not. All 29 students met the requirement for PEC 6.1. PECs 2.2 and 3.2 were assessed in BME 174 (Biomedical Regulatory Requirements). A composite performance indicators (PI), consisting of four factors was developed by the instructor, and used for assessment, with the target being that 95% of the class would be above the “unsatisfactory level. With respect to PEC 2.2, the only PI where the attainment was below the desired level was “Writing Test Protocol”, with only 48% of the students meeting the target attainment level. Corrective actions have been identified.

For PEC 3.2 the PI consisted of five factors. Achievement for four of the five factors exceeded the desired attainment level. However, achievement with respect to “Knowledge of World Regulations” fell far short of the target, with only 37 percent achieving the desired attainment level. In BME 177 (Physiology for Engineers), all but one of the 58 students enrolled achieved the desired attainment level of 70%. For BME 198A and BME 198B (Senior Project), the average ratings for each of the evaluation criteria are all around 80%, with very little variation between the two years. However, when the averages are compared with the median values, it can be seen that the median values are either very close to the average values or somewhat higher. This indicates that while most of the students are performing at a reasonably high level (80% or higher) there are a few students who are performing poorly, thus reducing the average values. It is clear that more effort needs to be put into raising the level of the students on the low end, especially those whose performance ratings were below three.

8. Proposed changes and goals (if any)

Addition of problem-solving and active learning components has shown some promising improvements in BME 115. In future semesters, further expansion of these components in BME 115 and introduction in other courses is expected to improve attainment levels for PEC 1.2. In order to improve achievement levels for PECs 2.1 and 2.3, the number of courses that offer laboratory components will be progressively increased. Contextually, a number of these lab components will be redesigned to include data acquisition from living systems, as recommended by ABET.

Part C Proposed Changes and Goals Status Update Increase the number of required courses offering laboratory components

BME 177 and BME 175 will have laboratory components starting Fall 2016

Revise lab components to include data acquisition from living systems

BME 115 labs revised to include data acquisition from living systems, effective Fall 2016

Add problem-solving and active learning components to BME 115 and BME 117

These components have been designed and will be implemented beginning Fall 2016.

Appendix A

Table A.1. Summary of Level of Attainment in Student Outcomes and Performance Evaluation Criteria

Student Outcome Performance Evaluation Criterion

Assessment Level

Results of Assessment

BME 115 Foundations of Biomedical Engineering

1.0 Ability to apply knowledge of mathematics, science and engineering

1.2 Apply conservation laws to biological and medical systems to solve biomedical engineering problems.

2

Homework assignments and exams used for assessment. 50% was deemed an acceptable level of attainment, because time pressure is an issue with the exams.

29 Total Students Midterm 1 Midterm 2 Midterm 3 Final Exam

>50% 18 24 20 27

>75% 7 8 6 15

>90% 3 2 2 1

2.0 Ability to design/conduct experiments and analyze/interpret data

2.1 Design and analyze appropriate experiments to measure or optimize specific engineering properties, incorporating statistical procedures

2

Assignments 2.1a, 2.1b, 2.1c employed for assessment. Assignment 2.1a, 89.6% of students scored 75% or higher. 79.3% of students performed at a level of 90% or higher. Assignment 2.1b, 89.6% of students scored 75% or higher. 62.1% of students performed at a level of 90% or higher. Assignment 2.1c, all students scored a 60% or higher. 89.6% of students performed at a level of 75% or higher and 48.3% of students performed at a level of 90% or higher.

2.3 Solve closed and open-ended bioengineering problems using experimental methodologies

2

Please see "Results of Assessment" for PEC 1.2 above. These two were assessed using the same assignments and examinations.

Student Outcome

Performance Evaluation Criterion

Assessment Level

Results of Assessment

BME 117 Biotransport Phenomena

1.0 Ability to apply knowledge of mathematics, science, and engineering

1.2 Apply conservation laws to biological and medical systems to solve biomedical engineering problems.

2

Homework assignments: Of 58 students enrolled, 81% scored an average of 75% or higher on homework assignments; 29% scored 90% or higher. The average score was 79% (standard deviation 17%). 70% is deemed an acceptable level for success in this category

37 Total Students Midterm 1 Midterm 2 Final Exam

>50% 58/58 40/58 50/58

>70% 43/58 8/58 29/58

>90% 9/58 1/58 5/58

1.3 Apply engineering fundamentals and scientific reasoning to model and predict responses at biological interfaces.

2

5.0 Ability to identify, formulate, and solve engineering problems

5.2 Simulate problems using mathematical modeling tools.

2

Homework assignments: ·86% of students submitting the assignments scored an average of 75% or higher. 76% of students scored higher than an average of 90% on all homework assignments in this category. Term paper assignment: 79% of students achieved a 75% or higher and 41% of students achieved a 90% or higher.

Student Outcome Performance Evaluation Criterion

Assessment Level

Results of Assessment

BME 173 Clinical Trials in Biomedical Engineering

3.0 Ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability

3.1 Demonstrate an understanding of the use of different biomedical devices in terms of safety and efficacy

1

Assessment was performed via grading of take home exam question 6, an analysis assignment, a standard operating procedure assignment and a group project A minimum of 70% required for meeting criterion. Of the 29 students enrolled, all but 4 were assessed to have met the

criterion.

6.0 Understanding of professional and ethical responsibility

6.1 Formulate and address ethical issues which arise in solving engineering problems and in the workplace

2

Assessment was performed via grading of take home exam question 6, a device analysis assignment, a Corrective and Preventative Assessment assignment and a group project A minimum of 70% required for meeting criterion. All 29 students enrolled in the class were assessed to have met the criterion.

Student Outcome Performance Evaluation Criterion

Assessment Level

Results of Assessment

BME 174 Design and Quality Requirements

2.0 Ability to design/conduct experiments and analyze/interpret data

2.2 Analyze and interpet results of specific and mandatory FDA testing

3

Four Performance Indicators (PI) were identified with attainment levels defined as Exceptional (4), Competent (3), Developing (2), Unsatisfactory (1). It was targeted that at least 50% of the students achieve levels 3 (competent) and 4 (exceptional) and that at least 95% achieve above the unsatisfactory level. All four PIs exceed the 95% above unsatisfactory goal. Three of four PIs exceed the 50% target for competent and exceptional results. The Writing Test Protocol did not quite achieve the goal since only 48.1% achieved the competent and exceptional levels combined; corrective actions identified.

3. Ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability

3.2 Demonstrate a knowledge of the contraints around process/product design based on FDA regulations

2

Five Performance Indicators (PI-5 through PI-9) were identified for assessment of attainment. It was targeted that at least 50% of the students achieve levels 3 (competent) and 4 (exceptional) and that at least 95% achieve above the unsatisfactory level. Two of the five performance indicators, PI-5 and PI-6, both for group assignments, exceed the 95% above unsatisfactory goal. It is notable that the individual performance on the risk management concepts, PI-7, was lower than during group work. Four of five performance indicators exceed the 50% target for competent and exceptional results. The PI-9, Knowledge of World Regulations did not achieve the goal since only 37.7% achieved the competent and exceptional levels combined.

Student Outcome

Performance Evaluation Criterion

Assessment Level

Results of Assessment

BME 177 Physiology for Engineers

1.0 Ability to apply knowledge of mathematics, science and engineering

1.1 Identify the basic structural and functional principles of human organ systems including repair systems

1

Based on write ups, presentations, and midterm and final examinations, of the 58 students enrolled in the class, all but one attained the required 70%.

BME 198A & 198B Senior Project

2.0 Ability to design/conduct experiments and analyze/interpret data

2.1 Design and analyze appropriate experiments to measure or optimize specific engineering properties , incorporating statistical procedures

3

Assessments in this category are an aggregate of several assignments identified in Table 4.6

Average 80.4%

SD 16.7%

% > 50% 93.1%

% > 75% 75.9%

% > 90% 27.6%

2.3 Solve closed and open-ended biomedical engineering problems using experimental methodologies

3

Assessments in this category are an aggregate of several assignments identified in Table 4.6

Average 76.3%

SD 18.8%

% > 50% 82.8%

% > 75% 72.4%

% > 90% 20.7%

Student Outcome

Performance Evaluation Criterion

Assessment Level

Results of Assessment

BME 198A & 198B Senior Project

4.0 Ability to function on multi-disciplinary teams

4.1 Function effectively as both team leader and team member in accomplishing engineering team projects 2

Assessments in this category are an aggregate of several assignments identified in Table 4.6

Average 86.5%

SD 10.4%

% > 50% 100.0%

% > 75% 86.2%

% > 90% 48.3%

5.0 Ability to identify, formulate, and solve engineering problems

5.3 Evaluate the constraints in a biomedical engineering problem and develop solutions

3

Assessments in this category are an aggregate of several assignments identified in Table 4.6

Average 80.4%

SD 16.7%

% > 50% 93.1%

% > 75% 75.9%

% > 90% 27.6%

7.0 Ability to communicate effectively

7.1 Communicate effectively in informal team settings and through formal and informal presentations, in written and oral formats

3

Assessments in this category are an aggregate of several assignments identified in Table 4.6

Average 81.1%

SD 15.2%

% > 50% 96.6%

% > 75% 75.9%

% > 90% 27.6%

Student Outcome

Performance Evaluation Criterion

Assessment Level

Results of Assessment

BME 198A & 198B Senior Project

9.0 Recognition of the need for and an ability to engage in life-long learning

9.1 Conduct a thorough information search, be resourceful in uncovering information, and critically evaluate information.

3

Assessments in this category are an aggregate of several assignments identified in Table 4.6

Average 78.5%

SD 14.3%

% > 50% 89.7%

% > 75% 79.3%

% > 90% 13.8%

10.0 Knowledge of contemporary issues

10.1 Compare and evaluate current and emerging biomedical engineering technologies 2

Assessments in this category are an aggregate of several assignments identified in Table 4.6

Average 80.0%

SD 17.0%

% > 50% 93.1%

% > 75% 86.2%

% > 90% 13.8%

11.0 Ability to use the techniques, skills and modern tools necessary for engineering practice

11.1 Apply appropriate software, modern tools, and techniques for design and analysis of biomedical systems

2

Assessments in this category are an aggregate of several assignments identified in Table 4.6

Average 77.4%

SD 20.6%

% > 50% 93.1%

% > 75% 82.8%

% > 90% 0.0%