Changing the Assessment Process in Mathematics

for Students in Engineering

P. V. Georgieva* and E. P. Nikolova**

Informatics Department in FCSE, Burgas Free University, Burgas, Bulgaria * penka.georgieva@bfu.bg, ** enikolva@bfu.bg

Abstract - Mathematical subjects studied in the Faculty

of Computer Science and Engineering at Burgas Free

University are focused on acquiring fundamental knowledge

and developing essential skills required for future engineers.

This paper looks into changing the process of assessment of

students’ achievements in mathematics from closed-book

exams to diversification of tasks and expected outcomes.

Three aspects of assessing the mathematical knowledge in

engineering programs are considered: the importance of

identifying the entry level, blended learning and creating

students’ portfolios with MatLab. The experience of the

authors in using the tools of blended learning and the use of

a software environment are presented. Some emerging

problems in the educational process that need to be faced

are outlined.

Keywords - mathematical knowledge assessment; blended

learning; students’ portfolios

I. INTRODUCTION

The key goals of studying mathematical subjects in engineering bachelor's and master's degrees were unchanged for the last century and they are: acquiring fundamental knowledge and developing essential skills that are required for future engineers. Achieving these goals today is impossible by following the old-fashioned methods of admission, training and learning such as:

• for enrolling in engineering studies, applicants have to pass a difficult mathematical entrance exam with the presumption that strong mathematical knowledge somehow predetermined the high potential of engineering skills;

• “standard” lectures: the professor “reads” the lecture, and the students “listen” and eventually write down notes in almost complete silence;

• “standard” mid-exams: the lecturer offers several problems to be solved and the students are expected to write down the solutions on paper, with the requisition that all the necessary knowledge must be pre-memorized;

• each of the math courses ends with an exam

procedure, in which firstly problems are solved as described above, and only those students who have shown the knowledge and skills to solve more than 50% of the problems correctly, pass to the second stage, in which they have to write on 2 or 3 topics from the exam questionnaire (again pre-memorized) i.e. the students sat closed-book exams.

But what was going on in the period between the lectures and the exam was not very important and/or interesting for the lecturer. What is more, after the mathematics exam, the future engineer was free to forget the lemmas, theorems and propositions, as well as their proofs.

Although the authors classify these approaches as old-fashioned, they have had produced generations of capable engineers, thanks to which engineering and technology have received such rapid development. Yet one fact has to be taken into account - in the previous decades, the engineering sciences remained steady and stable for a decade or two (in some areas).

Now the conditions, the requirements and the realities have changed substantially and each university is adjusting methodologically.

Firstly, the number of applicants for engineering majors has dropped significantly (Figure 1). For example, the decrease of engineering students in numbers for the last 10 academic year is almost 35% [1]. The number of students enrolled in engineering programs at BFU has been decreasing approximately by 40 % for the last 20 years. This leads to reduced competition at the entrance, where students with lower-than-average achievements enroll and begin to study in engineering programs.

Secondly, engineering knowledge itself is much more dynamic, rapidly evolving and thus requires different

This research is funded by National Research Fund of Bulgaria under

Contract No. KP-06-COST-8/ 06.08.2019 for providing national co-

financing for the participation of Bulgarian teams in approved actions

under the European program for cooperation in the field of research

and technology COST under the project "Characteristics prediction

and optimization of a photovoltaic system with artificial intelligence

methods”.

0

5 000

10 000

15 000

20 000

25 000

30 000

35 000

40 000

45 000

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 Figure 1. Number of students studying Engineering degrees in

Bulgaria for the period 2000-2018

MIPRO 2020/EE 1837

tools, technologies and skills to be acquired. For example, engineering students no longer draw by hand but use a wide variety of CAD/CAM software systems. BFU is not an exception and the discipline "Fundamentals of Design and AutoCAD" has been introduced for all engineering majors at FCSE since 2016. What is more, new specialties are emerging in the engineering field: robotics, IoT, and more. In response to current tendencies two new master degree programs have been introduced in FCSE: Artificial Intelligence and Robotics, and Electric Cars.

Thirdly, the students show little willingness to learn theoretical concepts. For example, the diagram in Figure 2 presents the results from the theoretical part of the linear algebra and analytical geometry exams (2015-2019). For this part of the examination procedure students are evaluated on a scale from 0 to 20 points. The chart shows that the percentage of students who received 15-17 points is below 17% (excluding academic year 2016/17), while the percentage of those within 18-20 points is below 12%. In addition, it is important to note that students’ knowledge is not systematized in a coherent scientific system, but is somewhat granular, detached, and slightly chaotic. They have easy access to information and knowledge from all aspects of science and life. If they don't know something, they simply google it. If they can’t do anything, they just watch a video (or two) and learn how to do it.

These facts also necessitate changes in learning, teaching and knowledge assessment. However, these changes in no way undermine the fundamental role that mathematical subjects play in the preparation and development of future engineers.

This article focuses on the transformation of the process of assessing the students’ achievements in university mathematics from closed-book exams to diversification of tasks and expected outcomes.

Three aspects of assessing mathematical knowledge in engineering programs are considered: the importance of identifying the entry-level; blended learning and creating students' portfolios with MatLab.

The authors' experience in using blended learning

tools and the use of a software environment is presented.

Some emerging problems in the educational process that need to be faced are outlined.

II. CHANGES IN LEARNING ENVIRONMENT

Challenging changes in the learning environment are widely discussed in academic circles. E. g. prof. Sh. Alexander focused on the rising cost of education and employability for graduates and how technology might be used for a more sustainable future in education have been discussed in the plenary of OEB 2019 Pushing Technological Boundaries. [2]

This paper is focused on active learning, the use of software systems for computer mathematics and internet in the learning process.

A. Active Learning

The concept of active learning is based on shifting from memorization of knowledge and facts (surface learning) toward understanding (deep learning). [3]

Four broad categories of instructional approaches in active learning have been identified: (a) individual activities, (b) paired activities, (c) informal small groups, and (d) cooperative student projects. These methods encompass many activities such as concept mapping, brainstorming, collaborative writing, case-based instruction, cooperative learning, role-playing, simulation, project-based learning, and peer teaching. [4]

Project-oriented learning (POL) involves students in various projects, usually leading to products [5, 6]. However, this process aims at the learning effect rather than the product itself. POL didactic strategy portrays active learning as an educational paradigm that transforms direct experience into a tool for supporting and stimulating learning. Key aspects of POL are: working autonomy, practical relevance, learning of soft skills, cooperation between university and practice.

In this context, engineering students in Burgas Free University are involved in small projects within individual courses, progressing to a final year project course. The projects are combined with traditional teaching methods within the same course with focus on the application. Projects are undertaken throughout the length of the course and vary in duration from a few weeks up to a whole year. Although POL is closely related to specialized subjects, there are possibilities for its application both in mathematics as a tool for engineering calculations and in the study of pure mathematics. Тhis is because POL, being directed to applications of knowledge through understanding, provokes strong interest in students. What is more, students receive both theoretical and technical knowledge and additionally soft skills such as working in a team, time and resource management, and last but not least - finding solutions, arguing and defending their opinion and position. [7]

B. Using Software Systems for Computer Mathematics

Many years of experience of the authors of this article show that lecturing on the whiteboard is an indispensable

Figure 2. Results from the theoretical part ot the exam in Linear

algebra and Analytic Geometry for the period 2015-2019

1838 MIPRO 2020/EE

methodological approach in teaching mathematics. The authors very much agree with and try to follow as much as possible Recommendation 4 of London Mathematical Society: Despite the agreed importance of modern, computer-based teaching and learning, lectures delivered using clearly visible boards should continue to play an important role. [8]

Nevertheless, the fact that easy access to powerful computers and software has led to significant changes in the teaching and learning of mathematics cannot be disregarded. Software packages that can be used as practical tools for the studying mathematics have been developed. MATLAB, MAPLE, and MATHEMATICA can support learning mathematics by performing complex symbolic calculations and transformations as well as for visualizing concepts and results. Statistical packages have also made a huge impact, allowing real-world data to be explored and analyzed. [8]

MATLAB is a software environment for performing various calculations (numeric and symbolic) and visualization of the obtained results. This software system consists of a kernel with built-in features that are dynamically complemented by toolboxes and additionally it has a built-in programming language that is capable of creating new applications. Key features of MATLAB are: accuracy in calculations, reliability of results, a large number of applications created in almost every field of science, the accessibility of applications (many are with open source code), vectorized operations, etc. These important advantages of MATLAB for engineering research are the prerequisite for its use in mathematical subjects at the Faculty of Computer Science and Engineering at Burgas Free University. [9], [10]

C. Internet in the Learning Process

Nowadays, using the internet for educational purposes plays an important role in both the teaching and learning processes. The benefits of incorporating internet in mathematics education are numerous. The Internet can be used as a methodological tool, e.g. video and animations are an effective way to illustrate geometric objects, vectors, etc. What is more, the students can contact their lecturers or/and other students almost continuously not only by emails but while working on assignments in web-based courses for assisted training by receiving comments and analytics. Internet resources help the students to explore new ideas and enrich their knowledge. Additionally, some online calculators visualize solutions step by step and thus are a useful tool in the learning process. Such are matrix calculators [11-13], derivative calculators [14-16], integral calculators [17-19], and others.

Changes in the learning environment necessitate changes in the forms of assessing knowledge and skills. The transition from traditional assessment forms to the diversification of the expected outcomes and how the lecturers in Mathematics at Burgas Free University proceed in achieving it are discussed in Section III.

III. FROM CLOSED-BOOK EXAMS TO DIVERSIFICATION OF

OUTCOMES

A. A Brief Overview of Assessment Practices

А preliminary clarification on the authors’ opinion is needed at this point. The authors of this paper believe that assessment is a part of the learning curve and its purpose is to inform and thus to improve both learning and teaching. The focus is on developing students' mathematical knowledge as well as their ability to evaluate themselves and improve.

The main types of knowledge assessment are as follows [20]:

(1) diagnostic assessment - aims at assessing students' knowledge and skills before the start of the course;

(2) formative assessment - focused on students' performance during the course and is done regularly throughout the semester;

(3) summative assessment - measures students' achievement at the end of the course;

(4) norm-referenced assessment - compares a student’s performance against other students;

(5) criterion-referenced assessment - measures a student’s performance against a goal, specific objective, or standard;

(6) interim/benchmark assessment - evaluates the overall student's achievements at the end of a grading period.

Assessment methods are the techniques and instruments for gathering information about the attained level of knowledge and skills. Subject learning outcomes include completing an action (describe, analyze, evaluate, solve, design or create) in the content of the scientific area, according to the context or professional environment or practice in which the student can apply the skills and knowledge. Here are eight important ways to measure students’ achievements in a university subject.

1. Tests - can be objective or subjective; multiple-

choice or free-response, true-false answers, matching

answers, filling blank space; written or oral; national or

locally generated.

2. Surveys - written or compiled through interviews. Students can be surveyed in courses (about the courses),

as they graduate (about the major), or as they change

majors (about their reasons for changing).

3. Evaluation reports - an individual or group is

evaluated through a checklist of skills and abilities.

4. Portfolios - collections of student work, usually

compiled for individual students under faculty supervision

following a standard departmental protocol. The contents

may be sorted into categories and by types, such as

homework, formal written papers, or examinations. The

work collected in a student’s portfolio should reflect the student’s progress through the major.

5. Essays and reports – focused on writing skills in

mathematics as well as knowledge of the subject matter.

MIPRO 2020/EE 1839

Essays and reports can be a part of courses and should be

candidates for inclusion in portfolios.

6. Oral presentations - demonstrate speaking ability,

confidence, and knowledge of the subject matter.

7. Practical assignment – students are involved in

project, research, discussion, modelling. 8. Dialogue with students – students’ attitudes,

expectations, and opinions can be a valuable element in the assessing process. Some methods for collecting data are student evaluations of courses, interviews held by faculty members or administrators, advising interactions, seminars, student journals, and informal interactions. [21]

Student cooperation and involvement are essential to most assessment methods.

Various alternative assessment methods have been

developed over the last decade [22], for example:

research skills in mathematics - the assessment

of students’ writing skills, oral presentations and

peer-reviewing of paper and/or report drafts;

designing test questions by the students - students are allowed to create own question and

answer based on the course materials;

presentation of a group work in the assessment -

as part of the assessment the students present

chapters from a book on group presentations;

group project which consists of two

presentations of the solution of an open-ended

problem set by the lecturer and an individual

written report;

weekly online quizzes;

presentations of applications of pure mathematics - the assessment consists of

creating a poster which illustrates an application

of math’s theory to real-world problems and

presenting it;

continuous assessment in a history of

mathematics module which is accomplished

through essay writing, peer-assessed posters for

mini-projects and the solution of a mathematical

question with the appropriate historical tools;

the use of an algorithmic e-assessment for

summative and formative assessments;

mathematical modelling project developed using a mathematical software package;

a written group project with individual

presentations of the same project;

presentations and quick quizzes on basic

material from the course;

Moore-method tasks- students have to find alone

answers to given problems without the help of

supporting material and present their solutions in

class.

B. Motivation for changing the assessment procedure

The starting point for designing the assessment procedure is to identify the area of knowledge, the appropriate depth and difficulty as well as the soft skills that the course will aim at. During the mathematical

training of future engineers at BFU, the lecturers' intent is the students to acquire additional competencies such as communication skills, creative thinking, team working, ability to deal with other people, decision making, problem-solving, personal character traits, etc. [23]

Closed-book exam is a traditional assessment method in which students rely entirely on their memory to solve problems, answer questions or write on a topic. It was exclusively used for assessing the mathematical knowledge of the engineering students at BFU in the 20th century. The exam consisted of solving problems and for those who met the criteria for a positive result, a written theoretical examination (writing definitions, theorems and proofs on predefined topics) was held.

Later, in some mathematical courses, the closed-book exams were substituted with tests. Over the years this assessment method proved to be unable to evaluate students' mathematical knowledge and skills neither as a system nor in detail. Gradually the tests were replaced with mixed assessments.

Since 2000, the evaluation of the academic achievements of students at BFU has been regulated by the BFU Students' Knowledge and Skills Assessment System. One of the basic principles of the system is the systematic nature of the students' efforts throughout the learning process through the use of effective credit accumulation mechanisms. The point scoring system requires ongoing monitoring of the knowledge acquired during the semester. Current monitoring is realized by two or three midterm tests, individual exercises in the electronic training platform and individual/team coursework.

Key principals for diversifying the tasks that form the final mark are:

formative assessment – the point-based mark system is introduced in many universities and

thus the assessment is based on an ongoing

assessment by monitoring students' progress and

providing feedback on how to improve their

learning process and a summary assessment that

assesses how well the students have performed

against the assessment criteria selected;

assigning weights to different activities – allows

the students to learn gradually and make

decisions on when to complete which tasks;

feedback - students receive feedback on their achievement throughout the academic year and

thus are encouraged to work steadily and

continuously and thus to improve their

performance;

creativity - students are provided with the

opportunity to be creative in new ways of

studying and to develop a range of transferable

skills they need to successfully enter the labour

market. On the other hand, the diversified

assessment provides lecturers with an

opportunity for creativity in the design and development of assessment tasks.

1840 MIPRO 2020/EE

The final mark is complex and includes points from the current monitoring, completing the course work and from the exam with 20% gained from current monitoring, 10% - course work and portfolios, and 60% - from the final examination procedure. (Fig. 4)

IV. EXPERIENCE, RESULTS AND EMERGING PROBLEMS IN

ASSESSING THE MATHEMATICAL KNOWLEDGE IN

ENGINEERING PROGRAMS

Assessment of students' mathematical skills at BSU begins with an entry-level exam and continues in each of the mathematical disciplines during the training period. In addition to these skills, a number of other transferable skills are assessed through individual and group coursework. Web-based courses provide an opportunity for assessment during the semester. Development of skills for using software packages in solving problems, as well as the evaluation of the achieved results is a main goal in mathematical courses.

A. Identifying the Entry-Level

The entry requirements for engineering degrees reflect the fact that there is a substantial mathematical component. This is the key motivator for conducting an exam at the beginning of the first semester for determining the entry-level in mathematics for the last 10 years. The exam covers topics from the upper secondary education level required for university courses. Based on the results, students who did not perform well (less than 60% of the test points) are required to attend an optional mathematics course. At the end of this course, they take the exam again and after passing it they can proceed with higher mathematics subjects.

The results from high school math also show a decrease. Indeed, while in 1995-1999 80% of the students showed marks from high school diplomas in the interval [5.00, 6.00], in 2000-2018 80% of the students had marks in the interval [4.00, 5.25]. Another trend observed in BFU is skipping the first exam date and attending the exam later.

B. Blended Learning

Blended learning methods use the best of traditional classroom learning and e-learning. The role of assessment in the blended learning is to create engaging assignments and to monitor student progress.

During their education, the students of Burgas Free University may use web-based courses for assisted

training. BFU currently is offering 291 BA courses and 178 MA courses delivered via blended learning methodology (https://moodle.bfu.bg/). The courses are developed in accordance with the framework approved at University level:

o General information about the course: subject and

objectives of the course, for whom it is intended,

what students will know after passing the course,

what are the duties of all involved in the

educational process, what is the duration of the

course. o Assessment and evaluation methods, procedures,

and tools;

o Schedule of the consultations.

o Course material;

o Teaching aids (references to books or developed

guides, examples)

o Additional support materials - software products;

a list of internet resources related to the material,

and more. The ways in which BFU mathematics professors

incorporate blended learning principles to the classroom are:

o digital revision tasks- the digital training platform

is used to set up individual exercises for students

and to monitor the progress of each student

throughout the semester, as well as to identify the

weaknesses;

o digital assessments for feedback- each of the

courses provides students with self-assessment

tests;

o virtual consultations- BFU mathematics professors use the digital learning platform for

counselling the students.

C. Students Portfolios with MATLAB

Student’s portfolio is a collection of research papers, reports, tests, exams, case studies, video, essays, journals, self-evaluations, exercises, etc. in which the students present examples of their best work. The portfolio is collected throughout a program or subject and is assessed by faculty members.

Introducing this additional opportunity for scoring points is а result of the lecturer's drive to improve students' programming skills, to develop students' mathematical modelling skills, to show them ways to use their mathematical knowledge in various subjects, and to develop students' employability skills.

A variant of the students’ portfolio is the focused portfolio, which concentrates on a specific aspect of a mathematics course. In mathematical subjects at BFU, this aspect is the use of mathematical software package MATLAB. The authors of this paper, being lecturers, expect that students work on the assignments during the tutorials under the guidance of the tutor and/or individually as homework. They are provided with a template for presenting their assignments. The work, collected in the student's portfolio, reflects his/her progress through the main stages of the course and is

Figure 4. From closed-book exams to diversification of tasks

MIPRO 2020/EE 1841

reviewed 2 (for Calculus) or 3 times (for Multivariable Calculus) per semester. The emphases are on differentiation, integration and graphing. There is one task involving the transition from problem-solving to proving. An additional practical assignment for developing a mathematical model and present it as a group project is included as optional. As an illustration three typical assignments for including in the portfolios are shown below.

1) Draw the area D: x=y; x=3-2y2, calculate the

double integral ∫∫(y2-x)dxdy and interpret the result. (use

Matlab)

2) Define the integral creteria for series

convergence. Use it to the convergence of

∑1/(n*(ln(n))^(1/2)), n2. Using Matlab, visualize the first 150 partial sums of the series.

3) Let i(t)=Im*sin(ωt-φ) (sinusoidal current) and

u(t)=Um*sin(ωt) (sinusoidal voltage), where Im and Um

are the amplitudes of the current and voltage, ω is the

angular frequency and φ is the phase difference between

voltage and current. Find the average power. Graph i(t) and u(t) for Im=2, Um=3, T=π and φ=π/2.(use Matlab)

Several additional documents are included in the

portfolio. These include an entry profile, an entry

questionnaire, counseling forms, and advisor notes. The

entry profile contains information about the student prior

to enrolling at the BFU, such as the mathematics course

grades from high school. It also contains the entrance

questionnaire, the placement scores from the entry-level examinations in mathematics at BFU and the advisor

notesDuring

In the current academic year, the authors plan to add

student voice as an important addition to the portfolio.

The students will be invited to choose the types of tasks

to be included in their portfolios - homework, essays,

exam papers, written samples, independent project

reports, and may further open the door to students’

attitudes and feelings (e.g. self-awareness of their path –

“Where you started from, where you want to be, and

where you are now”). Our experience shows that the process of portfolio creation itself is essential in the

education of each student because it is focused on the

individual performance, not on the accumulation of every

possible information.

In our opinion, some emerging problems in

engineering education are to be faced in the next decade.

Three of them are outlined below. First, effective

adaptation for the first-year students who show fewer

academic skills and learning habits. Second, the almost

unlimited volume of mathematical applications decreases

the motivation of the students. And third, it is not always

certain that students complete their assignments themselves.

V. CONLUSION

This article presents the experience of mathematics

lecturers at the FCSE of BFU regarding the new trends in

the assessment of mathematical knowledge in

engineering programs using blended learning tools and

software environments. Attention is drawn to formative

assessment in blended learning, which actively engages

students in the learning process, builds peer and self-

assessment skills and provides an opportunity to evaluate

both academic achievement and transferable skills.

The authors of this paper are planning to implement

some elements of learning analytics and using their

experience in point-based assessment system and blended

learning, to focus on new effective approaches is learning

and assessment process.

REFERENCES

[1] https://infostat.nsi.bg/infostat/pages/module.jsf?x_2=164

[2] OEB 2019, Pushing the Technological Boundaries,

www.youtube.com/watch?v=KbYAezm7SiA&feature=youtu.be

[3] R. Ritchhart, M. Church, and K. Morrison, “Making thinking visible: How to promote engagement, understanding, and

independence for all learners,” CA: Jossey-Bas, 2011.

[4] A. Roehl, S. Reddy and G. Shannon, “The flipped classroom: an opportunity to engage millennial students through active learning

strategies,” JFCS, Vol. 105 (2), 2013, pp. 44-49.

[5] St. Bell, Project-Based Learning for the 21st Century: Skills for the Future, The Clearing House: A Journal of Educational

Strategies, Volume 83, Issue 2, 2010, pp. 39-43.

[6] А. Kolmos, “Problem-Based and Project-Based Learning,” in

Skovsmose O., Valero P., (eds) University Science and Mathematics Education in Transition, Springer, Boston, MA,

2009.

[7] S. Mollova, M. Zhekov, A. Kostadinov, and P. Georgieva, “Laboratory model for research on computer cluster systems,”

Proceedings MIPRO, pp. 1595-1600, 2018

[8] “Mathematics Degrees, their Teaching and Assessment,” London Mathematical Society, 19 Nov 2009, https://futurelms.

wordpress.com/2010/04/08/teaching-position-statement/

[9] P. V. Georgieva, “Systems for computer Mathematics and MOOCs,” Computer Sciences and Communications, Vol. 7 (1),

2019, pp. 125-131

[10] P. V. Georgieva, “Applying MUPAD in university Mathematics courses,” BFU ANNUAL, Vol. XXX (1), 2014, pp. 236-245

[11] https://matrixcalc.org/bg/

[12] https://www.symbolab.com/solver/matrix-calculator

[13] https://www.emathhelp.net/calculators/linear-algebra/matrix-calculator/

[14] https://www.derivative-calculator.net/

[15] https://www.symbolab.com/solver/derivative-calculator

[16] https://www.mathpapa.com/derivative-calculator/

[17] https://www.integral-calculator.com/

[18] https://www.symbolab.com/solver/integral-calculator

[19] https://www.wolframalpha.com/calculators/integral-calculator/

[20] B. V. Prasanthi and V. Ж. Inti, “Classroom assessment methods and tools: a review,” International Journal of Research and

Analytical Reviews, vol. 6, issue 2, 2019, pp. 94-97.

[21] B. Gold, S. Z. Keith, and W. A. Marrion, Ed., “Assessment practices in undergraduate mathematics,” MAA Notes No 49,

1999

[22] Mapping university mathematics assessment practices, University of East Anglia, Edited by Paola Iannone, Adrian Simpson, 2012,

https://www.birmingham.ac.uk/Documents/college-eps/college /stem/mapping-universities-mathematics-assessment-practices.pdf

[23] P. V. Georgieva and E. P. Nikolova, “Enhancing communication

competences through Mathematics in Engineering curriculum,” 42nd MIPRO, 2019, pp. 1451-1456.

1842 MIPRO 2020/EE