Claims of mass test rigging

2014-09-08 00:00

EXPERTS believe that the Education Department may be manipulating the annual national assessment (ANA) results, masking a maths and literacy crisis.

The ANAs are intended to provide a snapshot of what is, and is not, being understood by younger pupils.

“This information gives a feedback mechanism where actions can be taken that directly and intelligently intervene at points needing it most,” said Wayne Hugo, professor in the School of Education and Development at the University of Kwa­Zulu-Natal.

“But for data to be useful it has to be reliable,” said Hugo. “The data here does not seem to be reliable.”

Hugo said the 2013 data, especially at Grade 6 level, show up very strange patterns that are hard to explain by any normal educational process. “The patterns point to the possibility of massive upward manipulation of the results, and the added danger of obscuring what the problems are and how to address them.”

Asked if the results had been manipulated to get closer to the set targets set for Grade 3 and 6, Elijah Mhlanga, acting chief director: communications, Department of Basic Education, said the department “does not and has no reason to manipulate results of the assessment which is the department’s own initiative to identify challenges and strengths in performance in the system.”

“The problem with this claim,” countered Hugo, “is that the ANAs have set up incentives, where schools, districts, provinces, and the country as a whole, treat the ANAs, not as a diagnostic snapshot of the state of learning in our schools, but as a competition to improve results.

“If a school can show dramatic improvement, or come out as one of the top performers, then this can be used to gain advantage over local rival schools.

“This is not to say that the ANAs have been a total waste of investment,” said Hugo.

“Our schooling system is slowly improving and maturing, and establishing national feedback mechanisms within the system over time is a key indicator of just such maturation.”

The 2014 ANA takes place from September 16 to 19 when school pupils around the country in Grades 1 to 9 will write tests in Language and Mathematics.

In 2013 the target for an acceptable level of achievement in maths for Grade 3 was set at 58% and for Grade 6 at 52%.

In 2012, Grade 1 achieved 68%, in 2013 the figure was 60%; in Grade 2, 2012 was 57% and 2013, 59%.

But in Grade 3 there is a fairly dramatic jump from 41% in 2012 to 53% in 2013.

A similar pattern plays out on the way to Grade 6. In Grade 4 for 2012 and 2013 the average percentage mark is 37%. In Grade 5 it is 30% in 2012 and 33% in 2013. But Grade 6 sees a jump from 27% in 2012 to 39% in 2013.

A similar jump in percentages between Grade 5 and 6 is seen at regional level. For example, in Grade 5 in KwaZulu-Natal the acceptable achievement in 2012 was 16,7. It nearly doubles in 2013 at Grade 6 to 30,4.

This jump is reflected across all the provinces. In North West Grade 5 in 2012 was 9,1 in 2013 Grade 6 is 20,8 — fully a 100% jump.

According to Hugo these “miraculous jumps” need additional investigation as there is no reason within the current teaching reform process for the results at grade 3 and 6 level to show such massive increases in comparison to the nearest grades.

“Exactly where the manipulation is happening — at teacher, school, district provincial or national level — is unclear, but gross distortion has been the result; a distortion that disenables an intelligent and informed response to a national crisis.

“Even if a credible answer for these wild swings could be given, the net end result is that effective feedback about the functioning of the South African ­education system over time is disenabled.”

EDUCATIONISTS are sceptical about the Annual National Assessment (ANA) and even the Basic Education Department is not so sure.

In the executive summary of the report on the Annual National Assessment of 2013 it is noted that “there are a number of limitations relating to the current methodology and therefore the results must be interpreted in the context of these limitations”.

“They are warning that you have to be extremely careful,” says Edith Dempster, senior research associate in the University of KwaZulu-Natal’s school of education. “But then they go on to say ‘we have done better’.”

Given the caveats in the executive summary, how can one judge the results and the conclusions to be accurate and reliable?

Elijah Mhlanga, acting chief director: communications, Department of Basic Education, said the department had always put in the foreground the limitations that needed to be considered in interpreting these results.

Another rider in the summary states “the comparability of the tests from one year to another cannot be guaranteed, which implies that comparability of the results from one year to the other may not be accurate”.

However, said Dempster, while the department is “very careful in explaining that you can’t compare one year to another; they then go onto conclude numeracy is improving”. Dempster said because the ANA does not use standardised tests, you can’t compare one year to another because each year pupils do a different test. “Consequently you can’t use them to measure progress.”

Dempster said a standardised test is required and the questions must be kept secret so they can be used again.

According to Mhlanga, the department is “exploring robust testing designs that will address these limitations. Part of the design is to administer tests that will be used for diagnostic purposes and therefore are non-secure while on the other hand, using secure systemic tests that will be kept confidential so that they be used from year to year to measure changes in a more valid and reliable way.”

Such as the changes that occur in the average scores for Mathematics. Extrapolating figures from the 2013 ANA, Dempster found a dramatic decline in average scores for Mathematics from 61,8% in Grade 1 to 14,4% in Grade 9. “But what does that tell you?” she asked. “Does this mean they are marking too low or the tests are too difficult? You don’t know what it tells.”

Mhlanga said the ANA was not designed “to explain causal relationships or answer the question ‘why’ but to provide largely descriptive statistics and information that shows the status quo in terms of performance at a given point in time.”

Mhlanga said the department had “always fore-grounded the limitations that must be borne in mind in interpreting these large-scale assessment results” and that the ANA could “never be a once-off event …”

SOUTH Africa’s schooling system consists of three phases: a foundation phase from Grade 0 to Grade 3 where basic concepts are taught; an intermediate phase from Grade 4 to 6 where these concepts are implemented and the senior phase from Grade 7 to 12. These grades are compulsory up to Grade 9.

THE ANA system was instituted in 2011 by Basic Education Minister Angie Motshekga to determine pupils’ levels of competency in literacy and numeracy and address the school dropout rate.

When last year’s matric class began their school career in 2002 there were 1 261 827 pupils. By the time of final exams that number had fallen to 562 112.

In July, Motshekga said the ANAs would be done away with from Grades 1, 2, 4 and 5 and only implemented for designated senior grades. This year’s ANA will continue as planned.

According to a department spokesperson, the minister “is currently in the process of reviewing the administration of ANA with a view to strengthening the impact of the assessment on the efficiency of the system and enhancing the quality of learning outcomes. The final decision on how the assessment will be strengthened and which grades will be affected will be taken after discussions with the Council of Education Ministers.”

BOX ONE

The Annual National Assessment data for 2013 was obtained from almost seven million pupils from more than 24 000 schools and the results focus on the national and provincial performance of pupils in Grades 1 to 6 and 9 in Mathematics and Home Language as well as providing analysis of pupil performance according to gender and poverty measurements.

Join the conversation!

24.com encourages commentary submitted via MyNews24. Contributions of 200 words or more will be considered for publication.

We reserve editorial discretion to decide what will be published.
Read our comments policy for guidelines on contributions.

24.com publishes all comments posted on articles provided that they adhere to our Comments Policy. Should you wish to report a comment for editorial review, please do so by clicking the 'Report Comment' button to the right of each comment.

Comment on this story
0 comments
Comments have been closed for this article.

Inside News24

 
/News

Book flights

Compare, Book, Fly

Traffic Alerts
There are new stories on the homepage. Click here to see them.
 
English
Afrikaans
isiZulu

Hello 

Create Profile

Creating your profile will enable you to submit photos and stories to get published on News24.


Please provide a username for your profile page:

This username must be unique, cannot be edited and will be used in the URL to your profile page across the entire 24.com network.

Settings

Location Settings

News24 allows you to edit the display of certain components based on a location. If you wish to personalise the page based on your preferences, please select a location for each component and click "Submit" in order for the changes to take affect.




Facebook Sign-In

Hi News addict,

Join the News24 Community to be involved in breaking the news.

Log in with Facebook to comment and personalise news, weather and listings.