Notes from the Integrated Education Program (IEP) Conference

A little late, but below are my notes from the Integrated Education Program (IEP) Conference in Pretoria (6-7 February 2008).

Overall notes

The Integrated Education Program (2004-08) aimed to improve the quality of primary education by supporting programmes in teacher education, as well as school management and governance, in selected districts in the Eastern Cape, KwaZulu-Natal, Limpopo and the Northern Cape. The program was funded by USAID (approx. R155m) and implemented in partnership with the South African Department of Education. Overall goal: to improve learner performance in literacy and numeracy (in gr 3), and mathematics and science (in gr 6) in participating schools (the poorest of the poor in SA).

As a result of the program the average improvement in learner performance was 5%. This shows just how difficult it is to raise the average grade.

The director general in the Department of Education, Duncan Hindle, said that we need to:

• Shift resources from the FET to GET phases, especially Foundation Phase.
• Raise expectations amongst parents and society at large of the high levels of teaching that they should expect for their children, of what the service levels should be. He visited a school where at 11am on a Friday all of the teachers had left for the weekend. When he asked the parents if they were concerned about this, they didn't realise that this was not normal practice.

General comments from speakers and the audience:

• A culture of assessment has not taken root amongst teachers. Not enough regular assessments, e.g. testing, and recording of results.
• Language of instruction (LOI): Despite the language policy, there is confusion in the some schools as to whether they should teach in the local language or in English. Some of the schools said that the LOI for gr 3 was isiXhosa, but when the learners were given survey instruments in isiXhosa they could barely understand the documents. The learners managed to answer the English instruments. There is a need to clarify the implementation of the LOI policy.
• Teachers have to be held accountable for performance.
  At the same time, they must be given solid training and support materials.
• It is crucial to develop literacy skills in the Foundation Phase for cognitive development. This requires reading and extended or complex writing, which is more than a paragraph, ideally a page of writing.

Statistical analysis in SA Education

Dr Luis Crouch. See How to stem the Matthew Effect in education. Additional notes below.

Suggestion for improving education in SA:

• Make schools divulge their results. Universal external assessment might help.
• Ensure that schools understand what their expected grade achievement is.
• Popularise good teachers: those that arrive on time, that cover the curriculum, etc.
• He gave an example of a school in the EC where the teachers vote democratically on what they'll teach that week. Can't run a school system like that.

Mid-90s, almost no public statistical analysis by Apartheid government. Since then, explosion of stats, but how much have we learned? Further, most academics, NGOs and education departments are slow to take up and act on the sound research findings that have been produced in post-94 SA, .

Learner drop-out is something to consider. But a much bigger problem is whether those in school are actually learning anything. The average child in SA learns less than 97% of European kids.

Effect of socioeconomic status (SES): Within the group of low parental wealth there is a big discrepency in reading scores. The poor get highly variable levels of education.

To run a good school, need:

• Principal who is pedagogical leader and good manager.
• Teachers that follow curriculum.
• Learners that are disciplined.
• Accountability.

IEP learner achievement results

Ms. Carla Pereira, JET Education Services

Findings
Gr 3: Literacy
Overall improvement in project schools over control groups (using very basic descriptive statistics), though not a huge improvement.

Gr 6: Science
Sometimes in project schools and control schools the grades actually dropped over the course of the studies.

Performance by skills
She confirmed that literacy intervention has an impact on mathematics and science skills that have narrative components. If a learner can't read, he/she can't understand maths and science concepts that require reading.

IEP impact study

Eric Schollar provided a qualitative review of the impact of the IEP and also lessons learned about the actual implementation of the program, which are relevant to any large-scale educational intervention.

(Gain refers to the difference between the control and project groups for the pre-, mid- and post-tests.) The percentages given below represent the gain made by the one group relative to the other, not the actual test results. The summary of the IEP impact is as follows (in terms of gain):

• Literacy (gr 3): +4%
• Numeracy (gr 3): +11.3%
• Mathematics (gr 6): -0.1%
• Natural Sciences (gr 6): +2.7%

Gain for gr 6 is not significant, but is very strongly concentrated at gr 3 level. Overall, the average gain (gr 3 & 6) is +4.5%. Schools in KZN made the most impressive gains.

Distribution of impact
Of the test groups (127 schools), the distribution of impact (i.e. greater than 4%) of the IEP was as follows:

• Positive impact: 45.7%
• No impact: 26.8%
• Negative impact: 27.6%

So for a program such as IEP, which cost R155m, more than half of participating schools were either unchanged or left in a worse position by the program. It is hard to understand how this happens, but apparently this distribution is about average for large-scale educational interventions. The reason is that some schools are so dysfunctional that more resources provided (e.g. through the IEP) make no positive difference whatsoever. External organisations simply cannot impose managerial authority on those schools. Basically, only the school can save itself!

Analysis of the impact
Schools that made the highest mean gains did so as follows:

• Literacy: +35%
• Numeracy: +24%
• Mathematics: +12%
• Natural Sciences: +18%

In other words, in some schools the project group improved the literacy scores by 35% relative to the control group.
Schools with the lowest mean gains:

• Literacy: -18%
• Numeracy: -9%
• Mathematics: -10%
• Natural Sciences: -25%

Why was impact concentrated in KZN?
Only in KZN were all scores increased at post-test. Why? The following reasons were offered, all of which are relevant to any educational intervention:

• In KZN the learner workbooks were supplied to all schools. Assessment resource banks (ARB) were provided in all participating provinces but in KZN they were accompanied by common assessments (supported by the DoE).
• In KZN information from the external evaluation was regularly supplied and performance targets set for schools.
• In KZN detailed monitoring instruments were used.

Overall insights
The overall insights and lessons learned for educational interventions are as follows:

• Teachers are very strongly in favour of the classroom-level support they have enjoyed through the IEP. Many of them are still uncertain of the practical application of the planning and methodological principles of the OBE curriculum in classrooms. Independent service provider (ISP) field workers are popular partly because they can demonstrate these aspects in real situations. (This insight dispels the commonly held notion that teachers don't want outsiders coming into their classrooms and helping them to do their jobs.)
• Mr Schollar felt that one of the main causes of learner performance improvements is the provision of a syllabus supported by learner workbooks, together with common assessment tools.
• Systemic, rather than localised, assessment is vital for the SA educational system. Mr Schollar believes that one of the most important steps taken by the DoE since 1994 is to begin to implement across-the-board assessments. In some instances, local assessments pass learners to the next grade while those same learners fail national assessments.
• Because many teachers are "confused about OBE", IEP training in this regard is one of the most highly valued elements of the program. Quality of outcome was mainly due to the materials supporting classroom teaching.
• Teachers very strongly favour the provision of project materials that provide guidance to classroom planning, activity and assessment. Mr Firoz Patel, Deputy Director-General, DoE, who presented later, even proposed that the support materials provide minute-by-minute guidance on how to implement the NCS in the context of the OBE philosophy.

Closing remarks

Dr Nick Taylor, CEO, Jet Education Services

• Teachers perceive OBE to be something completely different to their previous understanding of teaching. So they feel that they can't teach OBE until they've been trained and developed to do so. But OBE is not something magical. They are still meant to be imparting knowledge. Need to move beyond dependency culture on training.
• Secondly, need to follow text books It's all in there, just follow the book.
• Because of poor teachers, learners are being socialised into a mentality of low expectations, low grades and low performance. The learners don't know that they should be pushing themselves harder, that they can achieve more.