Seminar on 2016 HKDSE Visual Arts SBA

1 24 Oct 2015

Programme

2

S6 SBA Mark Submission

SBA Moderation Mechanism

Observations from 2015 DSE Moderation

District Meeting & Trial Marking

Q & A

Visual Arts

3

2016 HKDSE

School-based Assessment System (SBAS)

S6 Submissions

Content of the Sample

4

Note:

Critical studies as piece(s) of work

should be submitted in full version.

Starting from the 2016

exam, the max. no of

extracted pages of

SBA portfolios to be

submitted will be

increased from 12 to

24 pages

2016 Assessment Sheet

5 NO Student Name, NO School Name and NO Teacher Name in the Assessment Sheet

2016 Assessment Sheet (after covering the information)

6 NO Student Name, NO School Name and NO Teacher Name in the Assessment Sheet

Network Schools Mark Submission

7

The SBA moderation is conducted on a school

basis.

For subjects offered through a network programme,

if students with the network are taught and

assessed by the same teacher(s) using the same

assessment standards in conducting the SBA, they

should be grouped into one single moderation

group in the moderation process so that their

moderated results are comparable.

Frequently Asked Questions (English only)

8

https://www.hkdse.hkeaa.edu.hk/HKEAA/content/conn/HKEAA

_UCM/path/Contribution%20Folders/web/Public/FAQ_zh_HK.

html#SBA

Visual Arts

9

SBA Moderation Mechanism

Moderation Mechanism

10

Method 2 (Moderation by Expert Judgement Supplemented with Statistical Techniques)

To Improve Validity and Reliability of the Estimations

Moderation by expert judgement is adopted for those subjects with small

candidature or that involve outcomes which are very different from those

assessed through the public exam. In the moderation process, each school is

formed as a moderation group. Standardization process should therefore be

conducted within a school to ensure consistency of marking standard. The

steps involved in moderation by expert judgement after final mark submissions

by schools are :

(i) Sampling Student Works and Process the Moderation School Sample Mean

(ii) Adjustment of Simple Average by the Use of Ratio Estimation

(iii) Sharing Information across ALL Schools via Bayesian Modeling

(iv) Set up of a Tolerance Zone and Determination of Moderated School

Performance Level

(v) Calculation of Standardized Raw SBA scores

(vi) Determination of School Spread

(vii) Determination of Individual Moderated Student SBA Scores

RAW SBA Scores

11

Each school

6 stratum

6 samples

(stratified sampling)

SBA Moderation Method

12

6 samples chosen by HKEAA (stratified sampling)

Review School Teachers’ Marking

On-Line submission (40M@) ONE Portfolio

13

SBA Moderation (1) Reviewing Portfolio Samples (Extracted Pages)

Portfolio

Assessor

SBA

Supervisor Assistant

Examiner

Portfolio Sample (Extracted Pages)

(artwork & research workbook)

Assessment

Score Sheet

…….……..

…..……….

……………

……………

…………...

Assessment Score

Sheets

Assessment Criteria/ Rubrics

Assessment Criteria /

Rubrics

March 2016

14

SBA Moderation (2) Reviewing Portfolio Samples (School Visit)

Portfolio Assessor

(Tertiary Lecturer)

April – May 2016

Portfolio Sample (Extracted Pages)

(artwork & research workbook)

Assessment

Score Sheet

…….……..

…..……….

……………

……………

…………...

Assessment Score

Sheets

Assessment Criteria/ Rubrics

Assessment Criteria /

Rubrics

15

SBA

Supervisor

SBA

Supervisor

SBA Moderation (3) Determination of High School Adjusted Mean

M-AD

Portfolio Sample (Extracted Pages)

(artwork & research workbook)

Assessment

Score Sheet

…….……..

…..……….

……………

……………

…………...

Assessment Score

Sheets

Assessment Criteria/ Rubrics

Assessment Criteria /

Rubrics

May– June 2016

Research Workbook

16

Artwork/ Critical Studies

17

Moderation among schools

18

Subject Mean

School Mean

Sample Mean

External Assessors

Moderation Mean

Ratio Estimate for School

Average

Bayesian Modeling

Tolerance Zone

SD Raw SBA scores

SBA Score after Moderation

19

Subject Score of Individual students

20 Note: VA Subject Score = Moderated SBA Score (50%) + Exam Score(50%)

Moderation Report (Example)

21 http://www.hkeaa.edu.hk/DocLibrary/Media/Leaflets/HKDSE-SBA-ModerationBooklet_r.pdf

Moderation Report (Magnitude of D)

22

Comment Magnitude of D

Within the expected range 0 D < 6

Slightly lower / higher than expected 6 D < 12

Lower / higher than expected 12 D < 18

Much lower / higher than expected D 18

Table 1: Implication of the comment on the mean

(D = difference between the means of the moderated and raw SBA scores)

Moderation Report (Magnitude of R)

23

Comment Magnitude of R

Within the expected range 0.78 < R<1.1

Slightly wider than expected 0.74 < R 0.78

Wider than expected R 0.74

Slightly narrower than expected 1.1 R < 1.2

Narrower than expected R 1.2

Table 2: Implication of the comment on the spread

(R = ratio of the standard deviations of the moderated and raw SBA scores)

24

Handling SBA Plagiarism cases

http://www.hkeaa.edu.hk/en/sba/sba_hkdse/index.html

http://www.hkeaa.edu.hk/en/sba/sba_hkdse_elective/dse_subject.html?25

25

Note: One Form for all SBA

subject in each year

Visual Arts

26

Observations from 2015 DSE Moderation

27

Based on the portfolios samples submitted by

participating schools, the overall performance was

generally satisfactory. The possible reasons include:

According to the moderated scores, 74.5% of schools

fell into the ‘within the expected range’ category this

year. A majority of the school teachers demonstrated

a high level of consistency and reliability in assessing

their students’ work. It showed that in general, the

judgment of school teachers was appropriate.

In some school, the raw scores of SBA were inflated

this year. The inflated scores were found mainly

among students with the highest scores in schools.

Teachers should accurately assess students’

performance according to the rubrics and performance

descriptors.

General Observation

The Quality of Portfolios (Artworks)

28

The overall performance of students this year was fair,

particularly in the quality of the art works.

The use of visual arts medium, the mastery of creative

techniques, and the expression of the visual elements

for artworks were judged to be inadequate and some

students were not sufficiently trained in the

fundamentals, particularly in drawing and painting

skills.

Their application of visual language and the creative

works of various visual styles were found to be

incompetent.

Some students may have spent too much time on

preparing substantial Research Workbooks and may

have ignored the basic painting and other visual arts

techniques.

The Quality of Portfolios (Research Workbook)

29

Most of the students’ Research Workbooks displayed

substantial contents.

Their art criticism and theme development demonstrated

thorough research and exploration.

The final artworks and ideas were integrated naturally

and consistently.

However, some of them were found to have focused too

much on devoting producing their Research Workbook,

so much so that it was in the form of a report

The Research Workbook should be the true record of the

natural development of ideas and the process of

creation. The form of a report which was not necessary.

Using of information

30

Some Research Workbooks were collections of a large

amount of copied materials and secondary sources

downloaded from Internet, without citations of any original

sources or original writers. Students should complete an

assessment task honestly and responsibly in accordance

with the stipulated requirements.

It should be reiterated that the HKDSE Examination

Regulations stipulate that a candidate may be liable to

disqualification from part or the whole of the examination,

or suffer a mark penalty for breaching the regulations.

Students can refer to the information leaflet HKDSE

Examination-Information on School-based Assessment

(http://www.hkeaa.edu.hk/DocLibrary/Media/Leaflets/SBA

_pamphlet_E_web.pdf) for guidance on how to properly

acknowledge sources of information quoted in their works.

31

The comments from teachers in the marking sheet

slightly increased this year. The feedback was specific

and could assist in the moderation process by providing

more information about the assessments of different

levels of students.

During moderation, it was found that many students with

extremely high or low marks did not receive any

feedback from their teachers, thereby creating problems

for moderation.

The comments from teachers

Unified Themes and Frameworks

32

It was also found that students from some followed a

unified theme and an identical framework for developing

individual portfolios.

These artworks revealed that students had not digested

the materials or reflected enough on their learning in the

development of their ideas.

Little effort was shown in exploring themes, interpreting

the artwork, experimentation and reflection on their

work.

The Presentation of Extracted Pages for

Research Workbook

33

*Noted: Irregular samples need to be re-submitted

Presentation Remarks

1 page to

1 page Number of extracted

page was increased

Irregularity Cases

34

Corrupted Files

Missing Artworks or Portfolios

Portfolios exceed pages

Miscalculation

Wrongly input marks

Wrong score sheet

Disclosing School, teacher or candidates information

Some recommendation

35

Correct File Format (40MB + PDF + View Orientation )

Clear Image

Good Order

With Teacher Remarks

Appropriate Presentation of Artwork

Readable

Finishing Level

Presentation Skill

Do Not Over-decorate

Visual Arts

36

District Meeting & Trial Marking

District Meeting & Trial Marking

37

There are 5 samples for trial marking

5 to 10 minutes for each sample

Use the portfolio assessor view to review the samples

Refer to the requirements and assessment criteria

specified in the HKDSE SBA Handbook for Visual Arts

You should mark the sample in a holistic way. You

should refer to the assessment criteria in the SBA

handbook P.7-8 (Chi-version) and P.34-35 (Eng-version)

Each sample has one Score Sheet. There are two

assessment items, research work and artwork. The

mark awarded by school is only a reference. The mark

range of each portfolio was from 0 to 50.

Samples for Discussion

39

Concept Arts

Group Work

Photo Work

Scanning Problem

Structured Work

Typed Report

Typed Work

2015 HKDSE Visual Arts –

Professional Development Programmes

40

Date Event Place

4, 5 Nov 2015 2:30 pm – 5:30 pm

Briefing Session on

2015 HKDSE Visual Arts

Examination

Conference Hall, 4/F,

Hong Kong

Productivity Council

Building

Run-down

• Briefing on assessment requirements, marking criteria

and general performance on Paper 1 & 2

• Report on irregularity cases of 2015 HKDSE • Considerations Towards the Reference Materials that brought

to the Exam Centres

• Report on the Extended Consultation on the MTR

Recommendations of the curriculum and assessment

of the subject

41

Revised Assessment Sheet for 2017 HKDSE VA SBA

• No need to ÷ 2

• Full mark is 30

• No need to ÷ 2

• Full mark is 30

• X 2

• Full mark is 40

The full mark of the portfolio is 100.

The full mark of the portfolio is 100

No change in the marks proportion

No decimal places needed

More convenience and accuracy in calculation

The revised SBA Assessment Sheet will be

implemented with effect from the 2017 HKDSE

Examination

The revised SBA Assessment Sheet will be

updated on HKEAA web-site later

42

Q & A

43

Thank You

44