Embed Size (px)
Citation preview
Moshe Kam* and Michael Lightner**
*Robert Quinn Professor and Head; Electrical and Computer Engineering, Drexel University
**Professor and Chair; Electrical, Computer, and Energy Engineering, University of Colorado, Boulder
Members of ABET Board of Directors
Engineering Accreditation: Challenges and Opportunities
22
Contact Information
Moshe Kam
Robert Quinn Professor and Head
Department of Electrical and Computer Engineering
3141 Chestnut Street
Philadelphia, PA 19104
33
Contact Information
Michael Lightner
Professor and Chair
Department of Electrical, Computer, and Energy Engineering
College of Engineering and Applied Science
UCB 425
Boulder, CO 80309
44
DISCLAIMER• The issues examined in this talk have been discussed by
the IEEE Educational Activities Board, but do not represent the views of IEEE or IEEE-EAB, but only those of the presenters
• Material is provided for illustrative purposes only
• Description of various rules and regulations are made in general descriptive terms and are not intended for operational or legal use
• Material is not purported to represent the official policy of any accrediting body or any other governmental or non-governmental agency
• These slides have been compiled to present a rather complete story. The talk at the conference will use a subset of these slides.
55
A Few Words about IEEE• IEEE is the largest multinational professional engineering
association in the world - 380,000 members in 160 countries- A 501(c)3 organization in incorporated in New York
• Originally concentrating on power engineering and communications, IEEE at present spans technical interests across the spectrum of technology- From nanotechnology to oceanic engineering
• In many respects IEEE has become “the steward of Engineering” but especially the steward of all areas intersecting electrical, electronic, computer, communication, power and energy engineering
6
Early Presidents
Alexander G. Bell Elihu Thomson Charles Steinmetz Frank Sprague
77
Why is IEEE interested in Accreditation?
• Because it is in IEEE’s stated mission
• Because accreditation has significant impact on the content of the curriculum in IEEE’s fields of interest- And hence on the future of the profession
• Because IEEE’s involvement introduces the voice of the profession and its practitioners into the decision making process of educational institutions
88
Why is IEEE interested in Accreditation?• IEEE considers accreditation a strategic
objective and supports accrediting bodies worldwide
- The IEEE BoD allocates funds and human resources to accreditation on an annual basiso About 500 volunteerso >$2M/year in direct expenditures in 2008
99
University Degrees
• Let us begin with a exploration of what it means and does not mean to get a university degree in engineering – specifically electrical
1010
What Does it Mean to Get a BSEE from a University?
• First – it means you were accepted into the school- Around the world acceptance ranges from
automatic to exceedingly difficult• Second – By the standards of the university you
successfully completed the requirements for gaining a degree
• Third – You will be associated with all the other graduates of the university and the reputation and accomplishments of the faculty and staff of the university
1111
What Does it Not Mean to Get a BSEE from a University?
• First – It does not necessarily mean that you are a good electrical engineer
• Second – It does not necessarily mean that you will be good in design
• Third – It does not necessarily mean you will be good in analysis
• Fourth – It does not necessarily mean you will be a good employee
• Fifth – It does not necessarily mean you are prepared for graduate education at any university other than that university – and it may not mean even that
1212
“But Our University is a Great University!”
• What can the previous slides mean? • We know our university is a great
university and has a great electrical engineering department.
• How do we know?- From the performance of graduates- From the reputation and research results
of faculty and staff• We do not know it is great because of
accreditation
1313
Outline
• Accreditation Myths• Accreditation in Engineering, Computing and
Technology- Definition, aims, uses and misuses, models
• Building new accrediting bodies in the early 21st Century
1414
Accreditation Myths
• Accredited Programs produced better graduates
- NO, accreditation is, often, a statement of meeting minimum requirements
- Accreditation does not, necessarily, address admission requirements
- Accreditation is/should be voluntary and excellent programs may not choose to accredited
1515
Accreditation Myths
• Employers care whether a potential employee graduated from an accredited program- I have rarely had an employer tell me this
was important or ask if the department was accredited.
- Universities with great reputation rarely rely on accreditation as a credential
1616
Accreditation Myths
• Accreditation means that the program is ranked highly- There are 296 ABET accredited Electrical
Engineering programs in the US- The ranking of programs and universities is
entirely separate from the accreditation process – in most of the world
1717
Accreditation myths
• Faculty members are highly engaged in the accreditation process and are deeply honored by being part of an accredited program- Rarely true- Accreditation is often one more requirement on
faculty timeoWhile they may care deeply about education,
innovation, better teaching and support for students, accreditation is an administrative activity – often with little perceived value
o Accreditation has limited impact on faculty salary, research, promotion and tenure, publications, and professional reputation
- Sometimes accreditation leads to more resourceso But this is rare
1818
So Why Accreditation
• We know the many things that accreditation is not – why do we have accreditation and what is its value?
• Make no mistake, accreditation is valuable
1919
Outline
• Accreditation Myths• Accreditation in Engineering, Computing and
Technology- Definition, aims, uses and misuses, models
• Building a new accrediting body in the early 21st Century
2020
A Broad Definition of Accreditation
• Formal recognition of an educational program by an external body on the basis of an assessment of quality
• An evaluation process in which an objective group (accrediting body) examines an educational program to ensure that it is meeting minimum standards established by experts in the field- The outcome of the process is binary: program is either
accredited or not accredited
2121
Value of Accreditation • Accreditation tells perspective students that a
program meets minimum standards• Financial institutions may only choose to provide
student loans for study at an accredited university• Students transferring between accredited programs
can have some sense of equivalence – see Washington accord
• Students with an accredited undergraduate degree from one country may/should receive better consideration in another country than a student from an unaccredited program
• Accreditation provides/forces a periodic consideration of educational programs and provides outside benchmarks and evaluation
2222
Professional Licensure
• While accreditation concerns an educational program, licensure concerns the individual
• Licensure may be a legal requirement in some engineering professions and in some countries/states
• In many locales students who did not complete accredited programs face difficulties in getting licensed
2323
Accreditation and Licensure
• In the US, students graduating from an accredited program can immediately sit for the Fundamentals of Engineering exam – the first step to licensure- Without an accredited degree there are
work/experience requirements before an individual can sit for the beginning licensure exam
• This connection between accreditation and the beginning of the licensure process is of significant value in many areas of engineering. - A great motivation for obtaining accreditation and
graduating from a accredited program in those areas
2424
Operational Definition of Accreditation by CHEA* (US)
• Accreditation in higher education is defined as a collegial process based on self- and peer assessment for public accountability and improvement of academic quality
• [Peers = group of peer faculty and staff, professionals, and public members]
• Peers assess the quality of an institution or academic program and assist the faculty and staff in improvement
* Council for Higher Education Accreditation
2525
Three Major Activities
• The faculty, administrators, and staff of the institution or academic program conduct a self-study using the accrediting organization’s set of expectations about quality (standards, criteria) as their guide
• A team of peers, selected by the accrediting organization, reviews the evidence, visits the campus to interview the faculty and staff, and writes a report of its assessment, including a recommendation to the commission of the accrediting organization
2626
The third step…
• Guided by a set of expectations about quality and integrity, the commission - reviews the evidence and recommendation- makes a judgment- communicates the decision to the institution
o and other constituencies if appropriate
2727
Challenges to the Traditional Definition (1)
• Should the accreditation be done by an “external body”?- Is it possible to conduct accreditation by peer groups
o E.g., peer institutionso This is how accreditation first started
• Should the result of accreditation be binary?- Some groups in Europe have called for providing evaluation
in four categories with respect to every criteriono Fails to meet minimum requirementso Meets minimum requirementso Exceeds minimum requirementso Excels in meeting this criterion
2828
Challenges to the Traditional Definition (2)• Should we strive to meet minimum standards rather
than achieving continuous improvement and excellence?
• Will the current system of accreditation be useful to industry in the long term?- The mobility of labor has challenged traditional credentials- The ECE industry had already rejected the licensing process
o E.g., the industrial exemption in the United States -While in most states engineers in industry are exempt from licensing laws for work done for their industrial employers, the engineer and the engineer's company may still be liable for unethical, unsafe, or illegal activities.
2929
Some Clarifications
• In the US various professions have accrediting bodies – engineering, law, music, journalism, medicine
• These professional accrediting bodies look at specific programs – e.g., different engineering departments – not the entire university
• In the US there are regional accrediting bodies that accredit the entire university. - Typically at the undergraduate level, but increasingly at the
graduate level- Often professional accreditation will require this
regional/general university accreditation
3030
Some Clarifications
• While, in engineering, we discuss accrediting a specific engineering program this does involve other parts of the university - Teaching of mathematics, physics, chemistry,
language, humanities, etc- Electives in other engineering and computer science
departments- Availability of shared resources
o Computing facilities, library facilities, labs, etc
• As a result, where possible, engineering colleges like to have all their programs on the same accreditation cycle- Simply reduces the work required
3131
Looking Forward: Traditions
• The trends we observe in accreditation will challenge traditional models
• It is unlikely that over-prescriptive accreditation models will survive
• It is unlikely that models that are based solely on minimum thresholds will survive
3232
The Constituencies of an Educational Program
• Past, present and prospective students
• Prospective employers
• Other bodies of higher education
• Licensing bodies
• Government
• The public at large
3333
The Various Functions of Accreditation (1)
• Provide constituencies of the educational program with a guarantee that an educational program… - meets (minimum) standards- continues to evolve in order to incorporate best
practices
• Put a stamp of approval on graduates – they are ready to practice- Raises the issue of First Professional Degree in
Engineering- Regardless of accreditation, this is a questionable
statement to make
3434
The Various Functions of Accreditation (2)
• Provide educational programs with opportunities for self-definition and self-reflection - and with feedback on program content and
direction
• Provide opportunities for continuous improvement of education programs
3535
Misuse of Accreditation
• Coercion- The process needs to be voluntary
• Disciplinary action
• Ranking and comparison of schools
• Controlling the school
• Serving the interests of one constituency on the expense of others
• Homogenizing higher education
• Control competition
• Limit enrollments
3636
Accreditation is not indispensable…
• Industry can replace accreditation by other mechanisms of quality assessment- University rankings by academic bodies- University rankings by the popular press- Internal lists of “acceptable institutions”
o This is common with major US technology companies
- Entry exams and interviews of graduateso This is common in the US, exams can be formal or informal
• If accreditation is to survive it should be- Pertinent - Transparent- Fair- Economical - Adaptive to the business climate
3737
The Basic Structure of the Process: Accrediting Body
• Accrediting body defines its accreditation philosophy and publishes criteria and process
• Accrediting body identifies and trains program evaluators
• Bodies that recognize accrediting bodies require proof of decision independence- The funding mechanism and accreditation
decisions should be independent
3838
A Word of Caution: Independence
• In several instances, IEEE observed loss of independence of accrediting bodies
• In all of these cases, the accrediting body was discredited- Schools preferred foreign accrediting bodies over the local
one
• It is not clear whether government controlled accrediting bodies will be recognized in the future by international accords
3939
The Basic Structure of the Process: Program
• Program studies accrediting body literature • Program collects required material and verifies
presumption of accreditability
• Program requests an accreditation visit
• Program gets organized to provide information to accrediting body and visiting team
• Self study
4040
Interaction Between Accrediting Body and Program• Mutual agreement on visiting team
• Agreement on dates and logistics- Within published guidelines
• Pre-visit communications
• Accrediting visit and preliminary reporting
• Post-visit communications
• Report preparation and determination of outcome
• Post-report communications – possible appeals
4141
Two important caveats
• The accreditation visit is supposed to provide “no surprises”- All concerns that program evaluators have on the
basis of submitted data are supposed to be discussed ahead of the visit
• On site visit focuses on the accreditation criteria and their implementation- This is not the time for ‘free advice’ or planning of
the program future by the visiting group
4242
Looking Forward: Implementation
• At present most accreditation bodies operate on 5-7 year cycles
o A series of “dramatic events” followed by long periods of low or no activity
o Elaborate visits requiring significant preparation
• It is possible to design a much simpler process that takes advantage of progress in information technology- Information is posted and updated continually- Visits are shorter
o focus only on the few items that do not require face to face interaction
4343
Selected accrediting bodies (1)
• Engineers Australia
• Engineers Ireland
• Canadian Engineering Accreditation Board of the Canadian Council of Professional Engineers
• France: Commission des Titres d'Ingénieur
• Germany: ASIIN
• Hong Kong Institution of Engineers
For additional details see www.Accreditation.org
4444
Selected accrediting bodies (2)
• Japan: Japan Accreditation Board for Engineering Education
• Korea: Accreditation Board for Engineering Education of Korea
• Malaysia: Board of Engineers Malaysia
• Mexico: Council of Accreditation of the Education of Engineering
• New Zealand: Institution of Professional Engineers of New Zealand
For additional details see www.Accreditation.org
4545
Selected accrediting bodies (3)
• Singapore: Institution of Engineers Singapore
• South Africa: Engineering Council of South Africa
• United Kingdom: Engineering Council United Kingdom
• United States: ABET
• China: Developing pilot accreditation aiming toward joining Washington Accord – organized through CAST
For additional details see www.Accreditation.org
4646
Characteristics of Accreditation (1)• Voluntary
• Performed by an external agency- Based on the locale of the program- Uses representation of all major constituencies
o Government inspection is not Accreditation
• Based on clear published standards
• Evaluative – not regulatory- It is not the place of the visiting team to provide the visited program
with detailed prescriptions and methodology
4747
Characteristics of Accreditation (2)
• Requires continuous maintenance
• Binary (at present, in most cases)
• Cognizant of program objectives and goals- One size does not fit all- Accreditation is not meant to homogenize the
education system
4848
What are the Factors That May be Considered?
• Content of the curriculum- Is there enough exposure to discrete
mathematics?
• Size and skill base of the faculty- Does a Computer Science program have
individuals who are trained in Artificial Intelligence?
• Morale and governance of the faculty
4949
Potential Consideration Factors (2)
• Facilities- Does the Microwave Laboratory in an EE
program have a Spectrum Analyzer?
• Admission criteria- Do the admission criteria ensure that
incoming students have the basic skills required to attend the program?
5050
Potential Consideration Factors (3)
• Support services- Do the program’s computing facilities enjoy
professional system administration?
• Graduate placement- Do the majority of the program graduates find
gainful professional employment within 6 months of graduation?
• Budgets and expenditures
5151
Looking Forward…
• Many of the traditional factors required a site visit for verification
• A model that considers most of the factors on line is now possible
• Accreditation may become continuous rather than a discrete event- Shift from reaching the minimum to continuous
improvement
5252
Level of Specificity
• The degree to which criteria are defined in terms of numerical goals or specific coverage methodologies
• The degree of specificity depends on the accreditation model- The general trend in the last 10 years is away
from specifics
5353
Level of Specificity: Examples
• Non-specific- A computer science program needs to show that graduates
were exposed to the principles of database organization and have used this knowledge in open- ended exercises and projects
• Specific- A computer science program will include at least 24 hours
on in-class instruction on databases which includes: database models (at least 3 hours); relational models (at least 1 hour)…
- A minimum of two 6-hour laboratory exercises on databases must be included. These exercises include…
5454
A Two-Tier Process
• Usually accreditation of engineering, computing and technology programs relies on a “general accreditation” of the institution
• Another accrediting agency ascertains compliance with laws and regulations, basic fiscal solvency, and preservation of human rights
• If the first tier is missing, the “technical” accreditation needs to include it- This may be a challenge for new accrediting bodies
5555
Most of the Work Does Not Involve the Accrediting Body
• Program must establish mechanism to collect data on its activities
• Program must establish mechanism to use data to reaffirm or reform its activities
• Program must undergo a thorough self-study- This is often the most important outcome of the
accreditation process
5656
Different Approaches and Styles of Accreditation
• The Minimal Model
• The Regulatory Model
• The Outcome-Based Model
• The Peer-Review model
• The Program Club model
5757
The “Minimal Model”
• Ascertains basic characteristics of the school and program- Often numeric and law-based
o Does the school satisfy basic legal requirements?o Does the school have enough budget, infrastructure and
reserves to conduct the program?
• Ascertains existence of the fundamental basics in the school and program- Physical conditions, size and skill base of the faculty,
coverage of basic topics in the curriculum
• Provides a prescription for a minimal core and very general parameters for the rest of the curriculum
5858
Reflections on the Minimal Model
• It is easy to install and maintain as long as it adheres to the “minimal” philosophy
• Not a bad way to start an accrediting body
• Does not encourage continuous improvement
• The biggest danger is “mission creep”- More and more requirements
5959
The Regulatory Model
• Requires strict adherence to a core curriculum- E.g., defines the minimum requirements for a Software
Engineering curriculum
• Specifies parameters for the rest of the curriculum- E.g., at least 6 credit hours of post WWII history
• Often involving direct prescriptions of curriculum and faculty composition- E.g., “at least three faculty in manufacturing are required if
the body of students exceeds 120”
6060
Reflections on the Regulatory Model
• Makes the accrediting process uniform and potentially fair- Criteria are unambiguous and often numeric
• Difficult to establish and update- Leads to endless strife over what the “core” means
• Relatively easy to maintain- The key to success is adherence to clear rules
• Was shown to stifle innovation and creativity in the curriculum- This was the philosophy of the pre-2000 ABET model
6161
The Outcome-Based Model
• Prescribes a “small” core and basic requirements
• Prescribes basic parameters for the goals of the program- But does not specify the specific goals of the program
• Focuses on the goals and objectives of the program- E.g., to maximize the number of graduates who continue to Medical
or Law school- E.g., to maximize the number of graduates who become program
managers in the construction industry
• Requires evidence of measurement of goals
• Requires evidence of using the measurements to feed a quality improvement process
6262
Reflections on the Outcome-Based Model• Provides for significant diversity in goals and objectives
- Very different from the regulatory model
• Puts a lot of responsibility and risk in the hands of the program leaders- E.g., some programs may try to achieve goals that are
unattainable
• Sophisticated and hard to evaluate- Very difficult to avoid complaints on inconsistent evaluations
• This is the basic philosophy of the current ABET EC2000 and TC2000 criteria- Though implementation does not always follow the philosophy
6363
A Word of Caution: Outcome-Based Accreditation
• While outcome-based accreditation is the most popular paradigm for accreditation, it is not problem-free
• The prescriptive nature with respect to course content can be replaced by a prescriptive process with respect to assessments
• Too much data may be collected and analyzed in order to prove that methods were assessed
• Adherence to the process by zealous program evaluators may cause strong disagreements about methodology- E.g., the debate about Direct Assessment
64
Goal Evidence
Graduates are ready to enter the workforce
Placement figures of recent graduates
Programs develops future leaders of industry
Statistically-valid evidence of leadership positions for graduates of the last 20 years
Programs provides the Province with computer scientists needed to accomplish national R&D goals
Evidence that graduates settle in the Province and maintain employment in Computer Science
Program serves students who wish to gain engineering background before they develop careers in medicine or law
Evidence that graduates turn to practice of law and medicine in statistically meaningful numbers
Collection of Data
65
Evidence Use of Evidence
Employer survey indicates graduates have difficulties using modern computing tools for control
Junior class in Control Systems now includes a section and lab on symbolic computation for control
Percentage of students who choose graduate school is dropping
Department commissions a survey of recent graduates to understand their new post-graduation profile
Graduation rates are dropping; exam grades in mathematics are dropping
Department institutes an “entrance exam” in mathematics. Low achievers are directed to remedial classes.
Use of Data for Improvement
6666
The Peer Review Model
• A coalition of schools organizes in group of peers - Schools select their peers - Members from other constituencies are added
o Government, Industry, professional associations
• The peer groups conduct the review in evaluator teams
• Model requires an arbitrator and facilitator- Ideally a professional association
6767
Reflections on the Peer Review Model
• This is the way accreditation was done in the US in the early 20th Century- E.g., Princeton and Johns Hopkins came to visit the College
of Engineering at Drexel University in 1904
• Difficult to organize
• Considered less confrontational and more collegial
• Risk a drift in the direction of unpublished mandates
• Risks clashes of philosophies and program rivalry
• Selection/acceptance of peers may be complicated
6868
The “Program Club” Model
• Group of peer institutions create a “program club”- Use a common website for communication
• Programs that wish to join create a website with requested information
• Programs report continually on progress and experimentation in education
• New ideas are discussed and tried by members of the club
• Few on-site visits (possibly during an annual conference)
6969
Reflections on the “Program Club” Model
• Continuous accreditation model
• Difficult to organize
• Considered less confrontational and more collegial
• Risks clashes of philosophies and program rivalry
• Selection/acceptance of peers may be complicated
• “High maintenance”
7070
Key to Success: Consistency and Transparency
• Criteria need to be clear and published- Ambiguities and potential for different
interpretations need to be called out and addressed
- Terminology needs to be defined
• Visit and reports should follow the criteria closely- Findings and conclusions should use the criteria
and the established terminology
71
Accreditation and Licensing
7272
Relationship Between Licensing and Accreditation
• Accreditation is provided to educational programs
• A license is provided to individuals- Graduation from an accredited program is
often a condition for licensing- Licensing exams are often based on model
curricula of accrediting bodieso Serious problem in the US since EC2000
7373
Engineering and Computing Licensing in 2007…
• Licensing of engineering and computing professionals is perceived to be “broken” in many countries- Not a needed credential in many disciplines- Poor enforcement- Further weakening due to massive redistribution of labor
among markets in the early 2000s
• One possible solution - the Canadian model- Licensing = graduation from an accredited program PLUS
testing on safety, professionalism and ethicso NOT on school material
7474
Licensing and the First Professional Degree in Engineering• At present there is debate in the US and Europe about
“the first professional degree in engineering”
• The US National Academy of Engineering and several European organizations appears to favor a Master of Science degree- In Europe this view follows adoption of the Bologna Process
• The NCEES* (USA) wants B.S. degree plus 30 semester credits
• No consensus among professional organizations
*The National Council of Examiners for Engineering and Surveying (NCEES) is a national non-profit organization composed of engineering and surveying licensing boards representing all states and U.S. territories. NCEES develops, scores, and administers the examinations used for engineering and surveying licensure through the United States.
7575
Outline
• Purpose• Accreditation in Engineering, Computing and
Technology- Definition, aims, uses and misuses, models
• Building a new accrediting body in the Early 21st Century
7676
Scope
• Building new accrediting bodies provides an opportunity to use about 80 years of experience with existing bodies
• Buy-in needed from- Professional Associations and leaders of the
profession- Academic institutions and the faculty- Industry, especially employers of engineers and
technologists- Governmental bodies and regulators
7777
Desired Final Outcome
• A fully functional stable accrediting body, operating with clear rules and regulations, and with a transparent and simple structure
• Reputation for independence in accreditation decisions
• Membership of the accrediting body in the appropriate mutual recognition accords
7878
Opportunities to learn from existing bodies…
• Structure and basic processes
• Criteria
• Methodology - especially self studies and outcome-based techniques
• Development of constituency coalitions
• Decision independence- “independence from any parent entity, or sponsoring entity, for
the conduct of accreditation activities and determination of accreditation status” (CHEA 2007)
7979
Opportunities to improve on the operations
of existing accrediting bodies
• Better use of information technology and automation
• A more continuous and smooth process
• Experimentation with less centralized models- The Peer Review and Coalitional models
8080
New opportunities for accrediting bodies…
• Development and provisions of tools for continuous reporting, assessment and improvement
• Creation and maintenance of a registries - of engineers, computer scientists, and
technologists
8181
Who should govern the accrediting body?• Professional associations• Academic institutions• Industry
• Institutions from the three sectors should be invited to become Members of the accrediting body
• Voting Members in the annual/bi-annual assembly of Members
• Governments should be invited to observe and advise
8282
Thank you!
Questions or Comments?
8383
Additional Sources
• Presentations in EAB workshops on accreditation- Esp. by Lyle Feisel, see www.ieee.org/education
• Public domain information provided by ABET Inc., EUR-ACE, the Washington Accord website, CHEA- Mostly from the organizations’ web sites
• J.W. Prados, G. D. Peterson, and L.R. Lattuca: “Quality Assurance of Engineering Education through Accreditation: The Impact of Engineering Criteria 2000 and Its Global Influence,” Journal of Engineering Education, pp. 165-184, January 2005.
• Prof. Dr. Dirk Van Damme (Ghent University, Belgium). Accreditation in global higher education. The need for international information and cooperation. Outline of IAUP approach. May 2000.
