The differentiated paper system adopted in the SEC exams has resulted in many students obtaining lower grades than they deserve in the physics exam, according to a study presented at a recent seminar.
The study, carried out in 2002 for an M.Ed. dissertation, revealed that a large number of students who were good enough to be choosing Paper II A opted instead for the easier Paper II B.
This meant that the highest mark they could achieve was a Grade 4, when they had the potential to do better.
The migration of these high-ability students to Paper II B also resulted in shifting the average standard higher in both papers, meaning that other candidates got lower grades than they would have obtained if compared to a lower overall standard.
The finding prompted the researcher, Jacqueline Pace, to say that the differentiated paper system is perhaps creating more problems by its existence than it is solving.
The Secondary Education Certificate system of exams, which is the responsibility of the Matriculation and Secondary Education Certificate (MATSEC) Board, became fully operational in 1994, replacing the GCE 'O' Levels offered by English examination boards.
The SEC exams are intended to cater for a wider range of abilities than the GCEs and to emphasise achievement rather than selection, by giving students the opportunity to demonstrate what they know rather than what they don't.
To achieve this, papers at two levels of difficulty are offered. Candidates are asked to select the appropriate level for their ability in each subject.
While Paper 1 is common to all students, the choice lies in Paper II, whose A Paper is aimed at more academically able students and has more demanding questions than Paper II B. Paper A also allows for higher grades to be obtained (Grades 1-5, as opposed to Grades 4-7 for students opting for Paper B).
Ms Pace's study recognises Matsec's success in negotiating a difficult transition from foreign to locally based examinations. She says that the steady growth in candidature is a measure of this success and may be partially attributed to the creation of Paper II B.
However, she recommends that in the long term the differentiated paper system should be abolished, although she added that the recent introduction of Grade 5 within the range of grades for Paper A "is a move in the right direction", as the number of Paper A candidates has since increased.
For her study, Ms Pace analysed the candidates' performance in the different parts of the physics exam, which is the most popular science exam taken by school-leavers.
"Evidence has piled up considerably to support the existence of Paper B candidates who are of Paper A ability," she said.
For example, from the scores of Paper 1 it emerged that at least 12 per cent of Paper B candidates ranked within the top 30 per cent of ability of all the candidates.
There were also large differences between the high and low achievers in Paper II B.
Another piece of evidence came from the script analysis of Paper B candidates who got a Grade 4: they showed they were of Paper A abilities.
Apart from the resulting underachievement by these students, the candidates for whom the softer Paper B is intended are also at a disadvantage.
Ms Pace explained that this was due to the crossover of high-ability students to Paper B, making it more difficult for the others to achieve the deserved grades because they are "pushed down by being norm-referenced with a significant number of high-ability students".
A similar phenomenon took place with Paper A candidates, who got relatively high scores in each component but "do not seem to attain the good grades they merit. The migration of high achievers from Paper A to Paper B ensures that in the former, one only finds the topmost able candidates. Comparing a candidate with the best can only result in a more difficult contest for all."
Ms Pace's conclusion was that "the differentiated paper system renders the process of grading the candidates defective and inequitable."
She also found that an over-emphasis on verbal writing skills, especially in Paper II A, favoured girls. Paper I, on the other hand, measured a better distribution of skills and was therefore gender neutral.
She thus recommended that the weighting of Paper I be increased from 42.5 per cent to 60 per cent of the final grade.
Ms Pace's study was presented at a dissertation showcase held recently to celebrate the 25th anniversary of the setting up of the University of Malta's Department of Education.