Official Report (Hansard)

PDF version of this report (244.9 kb)

Committee for Education

CCEA Functions, Funding and Historical Examination Errors

The Chairperson: Gavin, Adrian, Richard and Anne Marie, thank you very much for coming.  You are all very welcome.  Obviously, there is a huge number of issues, as you can see.  Apologies for taking so long to get to this point.  Gavin, thank you for the paper that you supplied.  In this field, there seems to be a lot of discussion and activity.  I hand over to you to make your presentation, after which members will pursue some questions.

Mr Gavin Boyd (Council for the Curriculum, Examinations and Assessment): Thank you very much for the opportunity to address the Committee.  Some of what I say will refer to what Caroline said during the briefing before this session.

The education sector here has, for many years, recognised the vital link between curriculum assessment and qualifications.  For that reason, Ministers decided to merge the Northern Ireland Curriculum Council and the Northern Ireland Schools Examination and Assessment Council in 1994.  That made sense, not just from an educational perspective but in operating effectively and efficiently, because it merged two non-departmental public bodies (NDPB) and pooled complementary talent and expertise.  That merger created an organisation with a single, consistent and regional approach to curriculum, assessment and qualifications.

The curriculum is what is taught in our schools, and the statutory curriculum is what the law says must be taught.  The Council for the Curriculum, Examinations and Assessment (CCEA) led a detailed programme of research into the Northern Ireland curriculum in the late 1990s.  That identified many of the serious shortcomings of the 1989 curriculum, which was introduced by direct rule Ministers and was modelled on the 1987 curriculum in England.

Following that research and a very detailed development programme, and supported by an intensive implementation programme, the revised curriculum became law in 2007.  The revisions included reducing statutory content and giving teachers much more flexibility to develop their own lessons while ensuring a common minimum entitlement for all; putting more emphasis on the development of skills, like communication, problem-solving and working with others; and setting out clear progression paths from early years through the key stages, which allow children to build knowledge and understanding and connect their learning across the curriculum.  The revised curriculum is acknowledged locally and internationally as being a success.  However, as the world moves on, so must the curriculum, so we will continue to keep it under review.

Of course, curriculum must be supported by appropriate assessment — we need to know how our children are getting on.  The types of assessment that we use must be consistent with the types of teaching and learning that we are advocating.  To that end, a new range of statutory assessment arrangements at the end of key stages is being developed, and will be implemented from September this year. 

One of the issues that clearly emerged in the curriculum research is the impact of assessment or examinations on what happens in the classroom.  When children move into post-primary education, the importance of the curriculum recedes as the emphasis on GCSEs and GCEs increases.  For an education system to succeed in its goals, qualifications must reflect and support the curriculum.  That is why Northern Ireland has always had its own examinations, and why it was such an enlightened move to bring together curriculum and qualifications in the creation of CCEA. 

All aspects of CCEA's work are vitally important.  The curriculum's objectives of maximising the potential of the individual and their contribution to our society and economy will not be met if the curriculum is poorly designed.  Examinations are a particularly high-profile activity.  CCEA delivers over 400 examinations every year, both GCE and GCSE.  Last year, there were approximately 370,000 unit entries and CCEA issued approximately 200,000 grades for general qualifications.  Approximately 5,000 markers, examiners and other support staff were involved in the process.  The time frame from the date of the last examination to the issue of results is approximately eight weeks.  In addition to the summer series, CCEA runs three other examination series and provides a number of vocational qualifications.  Logistically it is very demanding and, of course, it must be got right.

In an open system, where schools can choose from a range of providers, each of which is much bigger than CCEA, 75% of all GCSEs and GCEs in Northern Ireland are taken with CCEA.  That reflects the enormous confidence that local schools and teachers have in CCEA, and is confirmed in CCEA's latest annual customer satisfaction survey, which shows that the satisfaction rating remains very high.  CCEA invests heavily in a range of quality management systems that underpin all of the processes. 

Despite a huge attention to detail, not everything works perfectly.  Last year, there were errors in 0·5% of the examinations delivered by CCEA.  We deeply regret that and always work very hard to ensure that no candidate is ever disadvantaged.  The comments that Sean Rogers made earlier particularly resonate with us.  It is very hard to consider the stress that 16-, 17- and 18-year-olds are under at examination time, but, again, our schools know that issues that are raised with their local awarding body will be dealt with.  On Monday this week, I dealt with a query from an MLA about one individual candidate, and we are trying to deal with the issues that the MLA has raised.  I am personally dealing with that.  We have a very detailed feedback system to ensure that any concerns that teachers have about any of our questions are considered as awards are being made.

CCEA also has a statutory responsibility to ensure that standards of examinations and assessments conducted in Northern Ireland are recognised as equivalent to the standards of examinations and assessments conducted by bodies or authorities exercising similar functions elsewhere in the United Kingdom.  That is a significant challenge, but is being managed very successfully.  Our qualifications are valued by institutions and employers all over Europe.

I have one final comment.  The Programme for Government includes a number of educational targets, some of which relate to examination performance by young people.  For those targets to be meaningful, the education system, curriculum, assessments and qualifications must be in line with the Executive's plans.  Judging by a very recent comment in England, our Executive have a great advantage in having control over our education system through their oversight of curriculum development, assessment and those qualifications delivered by CCEA.  Thank you.

The Chairperson: Thank you very much for that, Gavin, and for the information that you provided for us.  Obviously, it would be remiss of us as a Committee not to start with an issue that has been going on for some time; that is, the issue in relation to mathematics and the errors that had occurred.  That issue has been around for some considerable time, and you are well aware of the detailed and extensive correspondence that there has been between this Committee and a member of the public on CCEA.  The action plan seems to still indicate that there is work that is still ongoing to resolve those problems.  I notice from the action plan that CCEA has recruited an additional independent mathematics specialist and that checks have been completed on the June 2012 papers, which, to date, are clear.  Can you give us an update on where we are at regarding all of that in the action plan, particularly around the issue of mathematics, which still seems to be a particular problem?

Mr Boyd: I will make a few comments and then ask Anne Marie, who is the director responsible for qualifications, to deal with specifics.  Last year, there was considerable correspondence with a member of the public, including to the Committee, the Department and CCEA.  As a result of that correspondence, the Department commissioned two independent inquiries.  One of them was in relation to processes employed by CCEA in developing questions for examinations, and the other was in relation to a specific question in one of last year's examinations.  I remind the Committee that the inquiry into processes found that CCEA's processes were robust, appropriate and rigorously applied. 

In the second inquiry into the specific question, the report suggested that the question could have been better worded.  CCEA accepts that the question could have been better worded, although it is interesting to note that the independent report contained a difference of view between two of its contributors on that question.  The Committee will be aware of the enormous amount of work that went on to ensure that no candidate was disadvantaged.  In publishing the past papers, which we do every year to help candidates to prepare for future examinations, we have changed the wording of that question, so the question that is shown in the past paper is not the same as the wording that appeared.  I am absolutely satisfied that the awarding process was carried out rigorously and appropriately last year, and a huge amount of work went in to ensuring that was the case.  Anne Marie Duffy will now talk about the detail of the action plan.

Ms Anne Marie Duffy (Council for the Curriculum, Examinations and Assessment): We have turned the actions in the action plan into a monitoring grid, so we now monitor progress against them.  In the two independent reports, 25 recommendations were made, and 20 of those are complete to date.  Two recommendations are the subject of clarification with the consultant, and the remaining three relate to the ongoing implementation of training and guidance provided to senior examiners and the process to safeguard operations when examiners are performing multiple roles.  Gavin referred to putting greater clarity around the question and the more helpful wording of that question, and we have issued that into the system to help teachers who are preparing candidates for the 2012 examination.  In accordance with that, we are bringing forward new and enhanced procedures from September 2012.  They will impact only on the papers that are still to be developed, which will be for 2014.  Up until that date, all the 2012 and 2013 papers will go through additional checks, including, as you said, scrutiny by a subject expert. 

A lot of the issues that were raised relate to the modelling assumptions that are made around the mathematics questions when examiners are setting them into real life contexts and trying to be aware of the experience that a 16- or 17-year-old will have.  Also, they may only have had 10 weeks of study on some of these very technical mathematic programmes such as mechanics or statistics.  That remains a challenge.

Mr Boyd: Virtually all the correspondence that we received around mathematics problems are around the issue of context.  You referred to the correspondence that the Committee received last year, and I have received further correspondence this year.  The correspondent is clearly an excellent mathematician.  I will put this in my sort of thinking.  If I were to ask Committee members how long a day is, the chances are that you would say that it is 24 hours.  You would be wrong, because a day is not 24 hours.  If it were 24 hours, we would not have leap years.  All I am seeking to set out here is that there is a context that is appropriate for 16-, 17- or 18-year-olds who may only have been studying a particular topic for three or four periods a week for 10 weeks.  That context, as teachers will recognise, is not necessarily the same context that would be appropriate for very advanced and very well-trained mathematicians.  That, clearly, is a struggle for us.

The Chairperson: We will stay on this point.  Is there a particular question on the mathematics before we move on?

Mr Lunn: I was going to comment that some days are longer than others, Chairman.

The Chairperson: Now that they have discovered some gene or something in Switzerland or somewhere, they will probably bring up some other theory, which will probably change as well.  I will not get dragged into that.  For me, a day is very long.  I know that.

Gavin, we appreciate the information that you have repeatedly provided on the issue that we raised.  We will move to another issue.  It seems as though a huge debate has started, and Michael Gove, the Secretary of State for Education, has entered into that debate.  Recently, he said that exams would be made tougher and that there would be years where performance would dip.  He also raised the prospect of a wider reform of the exam system.  In response to an investigation by 'The Daily Telegraph' in 2011, he said that the current exam system is discredited and needs fundamental reform.  Michael Gove has said that, at some stage, he will move back to O Level and the Certificate of Secondary Education (CSE).  That will have very implications for students from Northern Ireland.  Some time ago, we had correspondence from you on changes to GCSEs.  We have now moved to position where Michael Gove has made these comments.  During the earlier briefing, Caroline referred to the report that came out yesterday from the Education Committee of the House of Commons.  Taking into account CCEA's function and role as established in Northern Ireland, what is its current thinking and view on what is the best way forward for the pupils in Northern Ireland?

Mr Boyd: I can give you a CCEA perspective on that, but it has to be overlaid by a policy perspective, which is a matter for the Minister and the Executive.  In my opening remarks, I set out what I believe to be the vital importance of the connection between what we teach in schools and how we assess that and the overall Programme for Government.  If we are to succeed in our objectives of improving life and life chances for individuals and improving our economy and the state of our society, it is very important that we ensure that what happens in our education system is lined up appropriately to support that. 

We are in an interesting situation where, for a number of years, we have worked in the same qualification system as England and Wales.  There is a different qualification system in Scotland, and there are different qualification systems right across Europe.  Those will have grown up over time, but I am assuming that they will have been designed to meet the needs of those communities and the requirements of those Governments.  It is entirely possible to design and measure qualification systems against each other.  Young people from Northern Ireland go to university in Scotland, where they compete against young people who have come through the Scottish qualification system.  They go to university in the Republic of Ireland, where they compete against young people who have gone through the Leaving Certificate, and vice versa.  There are European systems and British Isles systems that are set up to measure the portability and transportability of qualifications, and we can deal with that.  However, there is, perhaps, a more significant challenge.  Bear in mind, the further you go through schooling, the more the qualifications shape your schooling experience.  I do not want to use the term loosely, but you will have heard the expression that teachers teach to the test.  Increasingly, as you get into your teen years, teachers work to ensure that young people will get the best possible results in their test and, therefore, take that on board. 

My argument is that we should start off by asking what the appropriate qualification system is for our society and our young people.  Once we have defined that, we can work out how to set it up to measure against other systems.

Chair, you are absolutely right:  if we move away from the system that we have had for the past 30-odd years, which is the same as that operating in Wales and England, it will have implications that have to be managed. 

Adrian might have a perspective from the Department.

Mr Adrian Arbuthnot (Department of Education): I think that the Minister has made it clear over recent months that he wants to take decisions that are in the best interests of the pupils and schools of Northern Ireland.  There was a case in point a few months ago when we had a consultation on linear and unitised GCSEs.  The Minister decided, as a result of the consultation, that he would provide schools in Northern Ireland with a choice and that both options would be open for schools here.  I am quite sure that, in any future proposed changes to the GCSE system, the Minister will want to take a similar line in doing what is right for Northern Ireland.

The Chairperson: This is where I think that we need to have a very open debate on this issue.  You may have seen me looking frantically for a piece of paper, but, as always, I do not bring all my pieces of paper, so it must be on my desk.  I was looking for the minutes of a CCEA council meeting.  What I saw is in the public domain; it is on CCEA's website.  If I remember correctly, one member of the council talked about being careful about how the changes to the qualification system were communicated.  This will have serious implications for pupils in Northern Ireland.  If you move from the current process, whether it is linear or unitised, and everybody else in England moves to a different system, our pupils will be at a disadvantage.  How are we going to have that debate?

Gavin referred to the responses to the consultation, and members will remember that we got all the information.  In fact, the Department was very good in giving us all the responses.  That was on 25 January this year.  Various opinions were expressed around that.  The situation has now moved on more definitively.  The Minister who is responsible for Education in England, Michael Gove, said that they are going to go down a particular road.  So, we have to ensure that we make the right decisions.  However, I cannot see where, at this minute, that discussion, debate, consultation or consideration is being given or being had in light of what Michael Gove said.  People might ask what relevance that has to us, given that we have our own devolved Administration and the Executive will set the policy.  However, it has huge implications for pupils who decide that they want to have access to English examinations.  It may also affect how their examinations in Northern Ireland will be treated as far as entrance to universities is concerned.  I want to be absolutely sure about that.  I do not see the Department having taken any decision.  This Committee has received nothing from it about this other than, I think, a note to say that it was aware of what was being discussed.

Mr Arbuthnot: The issue first hit the public airwaves about a fortnight ago, as a result of a leak.  The Department of Education had no advanced warning or knowledge of the proposed changes to GCSEs in England.  Our Minister is taking unofficial soundings to weigh up what different stakeholders think about the proposed changes, and, today, as we speak, he is meeting his counterpart in Cardiff to talk about a range of common interests, including the GCSE issue.  So, we are assessing the implications of the proposed changes in England.  I expect that there will then be a public consultation here in Northern Ireland to determine what the most appropriate way forward is for us here.  However, it is also important to bear in mind that whatever qualifications we offer to young people in Northern Ireland must be of a comparable standard to those offered elsewhere.  There is an important role for the regulators in the different countries to ensure that the exams that are offered match up to each other and that one is not considered inferior to another.  So, that is an important safeguard that is in place.

The Chairperson: Yes, but the problem is that CCEA is the regulator and the awarding body in Northern Ireland, and, from that Committee discussion that I referred to, it would seem as though CCEA has taken a view — this is for Gavin to answer — that it does not want Ofqual to come in and make an assessment of its process.  Why is that the case?  Why do we not want to ensure that we have always kept the focus of this on the best outcome, positioning our young people in the best possible place to ensure that they have the best possible advantage?  I notice from the discussion at the council meeting of CCEA that concern was being expressed about Ofqual and how it was not going to be allowed to come in and make an assessment of CCEA.  Is that the most appropriate way for us to proceed?  Can you clarify for us what the current position is on that?  I have still not had an answer as to what CCEA's view is on that.  This has been going on.  I appreciate what Adrian said about the leak two weeks ago, but the decision on GCSEs was taken in July 2011.  So, we are a year down the road in respect of the GCSE issue.

Mr Boyd: First of all, I apologise that I do not recognise the reference that you are making to the minutes.  I will need to go back and check that so that I can give you a full answer.

The Chairperson: I wish that I had brought it with me, and I could have given you the reference number.

Mr Boyd: I will go back and check it out. 

Ofqual has a very clear role.  It is the regulator for general qualifications in England, and it is also the regulator for vocational qualifications in Northern Ireland.  The role of CCEA in Northern Ireland is very close to the role of the Scottish Qualifications Authority, as it is also the regulator for qualifications in Scotland.  There is a good reason for that:  it is because of the expertise, using the same expertise across similar issues in relatively small systems.  However, Ofqual has a very important role in working with the regulators in Wales and Northern Ireland and ensuring consistency in standards across the three countries.  Anne Marie works with Ofqual on a very regular basis.  There is a very strong role there.  However, the relationship is essentially different.  Ofqual is an independent body in England reporting directly to Parliament.  CCEA is a non-departmental public body, which works under the direction of the Minister of Education. 

I will pick up on a couple of points that I think are relevant.  We do not believe that the research evidence supports a couple of the assertions that have recently been made in England.  In fact, on the day that Secretary of State Gove made some of his recent announcements, Ofqual published research showing that, actually, universities and employers had great confidence in A levels.  They had some issues about some aspects of A levels, but, actually, the research published by Ofqual confirmed our own view, which is that there is great confidence in the A-level system.  That is not what was reflected in Secretary of State Gove's public position.

Secondly, the Secretary of State in England has taken a particular view about modular examinations, but, actually, the research does not necessarily support his view.  The research indicates that, in some subjects, modular assessment is particularly appropriate.  The research does not support the assertion that modularisation of subjects makes them necessarily easier, or, indeed, harder.  There are some fundamentals that we have real difficulties with about research.

The other point that I want to make is that our Minister made a decision on modular GCSEs after widespread consultation, but the fact of the matter is that some of the English awarding bodies that offer GCSEs in Northern Ireland have indicated that they will not offer modular qualifications in Northern Ireland.  So, because of market pressures elsewhere, that choice, which our Minister felt was important, on the basis of a widespread consultation that he carried out locally, would be denied to our schools in other circumstances.

The Chairperson: Gavin, was a conscious decision made by CCEA some time ago to push the boundaries of the qualifications and make it about the ability or outcome for pupils in terms of grades so that we saw the grades increase over the last number of years?  Are you referring to the report that was carried out by the National Foundation for Educational Research (NFER)?  CCEA is a part of that, is it not?

Mr Boyd: We are aware of the report that you are referring to and the work that is going on there.

The Chairperson: The National Foundation for Educational Research clearly indicated that the qualification regulators found that, from 2008 to 2010, 77% of A-level qualifications and 83% of CCEA AS-level qualifications produced significantly higher grades than would be expected based on candidates' GCSE prior attainment.  I have a worry that that confirms what Michael Gove said about dumbing down and making it easier.  I am just worried that we are getting into an area now where, as the saying goes, doctors differ and patients die.  There is clearly a debate going on out there about the strength and validity of qualifications that young people have.

I had the same discussion with the vice chancellor of a university, who is very concerned about the quality and robustness of the students who are now going to that particular university.  I wonder if that is an indication of the weaknesses in the curriculum, albeit everybody says that the revised curriculum is a wonderful piece of educational provision.  Is it as a result of reducing the grade boundaries that produce the percentage of candidates achieving high grades in CCEA exams?  How does that leave us at the end of the process of ensuring that our young people have the best opportunities?

Mr Boyd: There is so much in that that we could spend a considerable amount of time, and I would love to —

The Chairperson: But, has CCEA?  That is the thing.  That is what is worries me when I read some of those things.  It is not just a throwaway thing that Gove is doing in London, or an inconsequential thing that we will work our way through.  It is monumental.  I saw from the same council meeting minutes that two papers were produced and presented to council members and CCEA has done some work.  Are those papers available to this Committee?

Mr Boyd: I will certainly look at making the papers available to the Committee.  There are a number of very serious points there that I want to deal with.  First, NFER produced detailed work looking back over the past 10 years.  That work showed that standards over time and outcomes over the past 10 years in Northern Ireland had been consistent and that the relationship with outcomes in England and Wales had been consistent over that time as well.  Our children have traditionally done better in examinations than their counterparts in England and Wales.  This is not the time and place to debate why that is, but it has consistently been the case.  The relationship between what has happened in Northern Ireland and what has happened in England and Wales is clear and consistent, and that is set out in the NFER report.

The Chairperson: The majority of those examinations were linear.

Mr Boyd: I would have to go back and check that.  Modularisation has increased over time, but, over the past 10 years, the relationship between what has happened here and what has happened in England and Wales has, give or take, been the same.  There has been an improvement in the top grades, but this is where I have to make a point:  I expect to see a marginal improvement in outcomes year on year because we make scripts available to candidates, and, through the chief examiner's report, we make information available to teachers.  We identify where candidates have performed less well, and we identify to teachers where they need to focus their efforts.  We are blessed with a very well qualified and a very highly motivated teaching force here, and, typically, we have more specialist teachers in the sixth form than you would expect to find in England.  Therefore, we are very well geared up, and our teachers have a tremendous track record of preparing our young people well for examinations.  All other things being equal, we expect outcomes to move up marginally year on year, particularly when the system is geared up to measure schools on examination performance.  That may be right or it may be wrong, but that is what we look at.

The second point that I will take up is the one that you made about the vice chancellor.  Recently, I spoke to the vice chancellor at Queen's about this issue.  We conduct examinations in a particular way.  Yes, we have modular examinations, but we also have more examination questions on the paper than was the case when I was doing examinations a long time ago.  Forty years ago, we tended to have three or four questions on a three-hour examination paper, and you took 45 minutes or an hour to write an answer.  Typically, they were long essays.  That is not the way that we structure examinations these days.  That is not the fault of young people or the teachers who prepare them for examinations.  When universities tell us that they want to see young people coming forward with better-developed skills for analysis or making a critical argument in a detailed essay, we can take that on board, but we have to do so by reshaping how we do our current assessment.  I cannot subscribe to the notion that our young people today are any less able.  In fact, I think that they are significantly better prepared for examinations than they were a generation ago.

The Chairperson: There are still a number of other issues, so I will open it up to members.

Miss M McIlveen: Ofqual has been mentioned, and it was set up to avoid an inherent conflict of interests with the functions in the Qualifications and Curriculum Authority (QCA).  Could the same accusation be made of CCEA in relation to its functions in that there may be a conflict of interest?

Mr Boyd: I do not believe that there is a conflict of interest, because the notion of a qualifications regulator in the way that Ofqual discharges those responsibilities is something that is almost unique to England, Wales and Northern Ireland.  It is almost unique in the world that schools have an option in where they take state examinations.  In most other jurisdictions in most other developed countries, a state examinations body delivers state examinations and has a responsibility for maintaining standards over time.  I believe that it is the model in England that is rather strange.

Miss M McIlveen: Do you not believe that there is any conflict of interest in CCEA?

Mr Boyd: We always have to ensure that we manage situations.  I have not had the chance to read the report from the Select Committee on Education in England, but I have read the summary.  One of the things it says very early in the summary is that it considered the option of going to a single examinations body but rejected that on the basis of the risks involved in managing the transition and the costs involved.  In fact, in Northern Ireland, we have the advantage of already having a state examinations body.

Miss M McIlveen: I will move on to the paper that you provided on funding.  Obviously, you have highlighted the fact that you see very effective budget management and that quite a considerable amount of money has been handed back.  The accusation could be levelled at you that that is a result of poor budget planning and that you have perhaps overestimated your need in the past three years.  Will you discuss some of the issues around why so much money has been able to be handed back?

Mr Boyd: The funding model essentially has three elements to it:  the core funding for running the day-to-day organisation, the generation of income from the sales of examinations, and the third element, which is really Richard's territory, effectively of project funding around curriculum and assessment development.

Examinations income has risen steadily over the past number of years.  From memory, we are generating £2 million more than we were five years ago in terms of examinations.  That is a function of two things:  price movements, and increasing numbers of schools choosing to take CCEA examinations.  It is not always possible to predict how much examination income there will be, but the good news is that, as soon as it is identified, it goes back to the Department of Education.

Project funding is always agreed in advance with the Department of Education, but it is entirely dependent on the pace at which projects can be developed and the uptake of, for example, training.  In 2011-12, for example, because of industrial action that was being undertaken by a couple of the teachers' unions, there was a significant drop-off in the need to pay teacher substitution for teachers to do training courses for the new statutory assessment.

Thirdly, CCEA has worked very hard to improve its operating systems.  That has included gaining the benefits of increased technology for examinations, which means that we expect to see the unit costs of delivering examinations to go down over time.  CCEA has invested very heavily in modernising systems, and over the past two years, for example, core staffing has reduced by 10%.

Miss M McIlveen: In relation to sales, is it true that you no longer sell examination papers to England and Wales?

Mr Boyd: The council took a decision some months ago to pull out of the English market.  There were very small numbers of our GCSEs and GCEs taken-up by schools in England:  about 5,000 GCSEs.  That was an attractive proposition when the specifications and the set-up of exams were essentially the same, because there was very limited additional cost.  However, as soon as we could see policy divergences starting to emerge, it was clear that the very small numbers of examinations that were taken-up by schools in England were not going to pay the cost of, for example, changing specifications in England for GCSEs and moving them away from a modular to a linear specification.  It simply did not make economic sense; it would be a drain on the local taxpayer.

There was a further consideration:  Ofqual was given powers to fine after some of the issues that arose last year.  Those powers to fine extend to about 10% of an awarding body's turnover.  In CCEA terms, that could amount to £950,000 or thereabouts, which was simply not an appropriate risk to take, given the very small income generated from qualifications in England.  The decision to pull out was largely driven by changes in policy, and we could not afford to develop new specifications, given the very small take-up of CCEA qualifications in England.

Miss M McIlveen: Did that recommendation come from you, or was there any political interference in relation to that?

Mr Boyd: Absolutely none.  A paper was developed by Anne Marie and brought to the council with a number of options in it, setting out the facts, and the council took a decision on that basis.

Miss M McIlveen: It was purely economic?

Mr Boyd: Absolutely.  There were emerging policy differences, but it was driven by economics.

The Chairperson: Just on that point, Gavin, there will clearly be a loss of money to CCEA as a result of that.  This is what concerns me about the commercial element of CCEA.  Are the decisions made in CCEA, first and foremost, on educational advantage, or are they made on the basis of loss of revenue?  You are an arm's-length body, so you must be accountable for your finances, and there have been issues and problems in the past around that.  We now have a situation in which there is going to be a loss of revenue.  What was the driving factor in coming to that decision?  Was it a commercial decision, or was it an educational decision?

Mr Boyd: CCEA has a range of statutory obligations in respect of Northern Ireland, and it satisfies those obligations.  There will be a potential loss of revenue of about £100,000 in a couple of years' time when the decision fully takes effect.  As accounting officer, my view is that I will have to look at that in terms of the attendant risks and the additional cost that could have been required to continue to operate in that marketplace.  As accounting officer on those issues, my assessment was that I could not justify the use of public money in those circumstances.

The Chairperson: If you had listened to the Minister for Regional Development today, you would have heard that he is convinced that being fined £90 for parking on a double-yellow line or in an inappropriate place is a greater deterrent than being fined £60.  The fear of being hauled over the coals by Ofqual in respect of a penalty would surely concentrate minds in any organisation.  That might not have led us to the situation that we have found ourselves in with regard to mathematics, because, clearly, that would have focused minds.

I am in possession of a letter that was sent by the Education Minister to one of my colleagues, who happens to be the Finance Minister.  He was obviously very exercised about the issue of the fines when he said that, while there will be a slight loss of income as a result of the decision, CCEA feels that the Ofqual power to fine would attract criticism that any failures by CCEA had cost the public purse in Northern Ireland, further reducing the amount of money available to front line educational services and damaging public confidence in the examination system.

Mr Boyd: I would probably react to that by saying that I regard the level of governance and oversight that is applied to CCEA through this Committee and through the Department as being of a considerably higher order than the activities that are taken on by Ofqual.  Therefore, in respect of all those performance issues that you talk about, I think that we are held to a much higher standard and much more direct control by the Committee and the Department.

Mr Kinahan: I want to change the focus slightly.  You mentioned at the beginning that it would be great to have the curriculum and the qualifications working together.  In a debate in the Chamber, we looked at the end result, which is getting jobs and employment, and whether any studies exist linking those to make sure that we are training people to get the right skills to get jobs.  One Member said in the Chamber — I cannot remember who it was — that any time anyone has tried to do that, it has been a complete disaster because it is second-guessing into the wind. 

Should we look further afield?  We have always been renowned as having an excellent education system here, but I feel that we are often too insular in Northern Ireland in that we are looking at training people for Northern Ireland jobs, and a little bit for Ireland, and a little bit for the UK.  We should be looking globally, particularly at the moment, whether that is in Australia, India or China.  We should be producing children who can go anywhere.  On the back of that, how do our exam passes rate outside the UK?  Are we respected and well thought of?  At the same time, are we looking enough outside the UK towards where we place children worldwide?  It is a global world, not a little island.

Mr Boyd: I will go right back to the curriculum review, which looked at best practice worldwide.  It looked very carefully at what our young people needed to equip them to deal with a changing world.  For example, it is less likely now that we prepare young people for a career.  It is more likely now that we prepare people for a portfolio of careers.  They must have that level of flexibility.

In the curriculum review, we identified the vital importance of skills such as communication, working with others and problem solving.  Those are precisely the sort of things that young people need.  There is a really interesting debate that we will have to have.  Secretary of State Gove has made it clear that he would prefer to see much more content in the English curriculum and English qualifications.  He has talked about the importance of great English literature, such as Shakespeare and Wordsworth, etc.  Those things are important.  However, when it comes to applying for a job or being successful in an interview or developing an economy, the other skills are going to be very important. 

The Minister has put great emphasis on science, technology, engineering and maths (STEM), and you will be aware of the initiatives around that.  I was looking at comments that Secretary of State Gove made in England with reference to looking at models of great world performance, such as Singapore, where the education system has been very successful.  Actually, in Singapore, they are just kicking off a review of the curriculum and qualifications system to try to prepare their young people to be more flexible and to be able to deal with the level of innovation and change that is coming down the line.  Therefore, it is a matter of content versus skills.  Sometimes, I hear employers saying that, when they bring in graduates, they cannot answer the phone.  We do not ask schools to teach young people how to answer the phone, but we are much better at developing a wider range of skills through not just standard qualifications, but all those other experiences that they get at school as well, whether it be sport, debating societies, choirs or whatever.

Mr Kinahan: Do you think that we are well placed worldwide?

Mr Boyd: The challenge for us is that the world is changing, and it is changing fairly rapidly.  That means that you have to keep things continually under review.  Some years ago, we identified a need to encourage more wealth creators and more entrepreneurship in Northern Ireland.  That needs to be reflected in curriculum and qualifications.  We have a very strong pharmaceuticals industry.  There are very specific needs to produce high quality young industrial chemists as well as pharmacists.  Therefore, we can tie those things together.  The argument that I would make is that it is important to tie those things together for our community, our economy and our young people.  We need to keep all those things under our control.

Mr Arbuthnot: I think it is worth adding that there are increasing numbers of business/education linkages across Northern Ireland between schools and local employers so that young people can, at a very early stage of their education career, understand the world of work and understand what employers might be looking for in future employees.  Therefore, making those linkages during school years is an important aspect of the preparation of young people for the world after school.

Mr Kinahan: I would like to see those linkages going further to other countries as well.

Ms Duffy: I would add that, recently, we brought forward a new portfolio of applied GCEs.  One of those is built on global industries, some of which work in Northern Ireland.  For example, one of the new GCEs that you will be aware of is in software and systems development, which has had Allstate, Momentum and Citibank, and a lot more involvement of business and industry, as we build applied qualifications for the 21st century, and they are very encouraging.

The Chairperson: I take that point on board, Anne Marie.  That is a valid point to make in regard to that particular qualification.  However, in the overall debate on whether we go linear or unitised, I have still not been able to get a satisfactory answer that convinces me that CCEA is taking a decision on purely educational grounds.  I still have not got any clarification on that. 

Members should revisit the consultation responses, where teachers say that continuous assessment causes too much stress for pupils and refer to the stress that pupils are allegedly under by doing other sorts of tests.  One of the responses says that continuous assessment causes too much stress for pupils at a time when they are only beginning to learn the skills that are necessary to achieve success at Key Stage 4.  It goes on to say that continuous assessment also raises expectations about exam outcomes, which can lower self-esteem and confidence if they are not met and that, with linear exams, pupils can revisit past learning and build their response skills over a couple of years and be better prepared for their exams.  Adrian was here when we presented all of this information, and another comment states that GCSEs in England are changing to linear, except for maths and science, which are still out to consultation, but, most likely, will also go.  The response expresses the wish that CCEA qualifications have parity with the exam boards in England to ensure that pupils are not disadvantaged when applying for university courses.  That is based on educational outcomes, not on whether it is financially advantageous to a particular organisation.  I still have not got an answer to that.

Mr Arbuthnot: The ultimate decision on what direction we go will be a political decision.  The Minister will take the decision on the basis of consultation and advice that he receives from a wide variety of sources, including CCEA.

The Chairperson: Adrian, with the greatest respect, according to the Minister, it was CCEA that made the decisions to do what it has done.  When the Minister was asked what were the two main factors that were influencing CCEA's decision, he said that those were the decision taken by Michael Gove, and the new powers that were announced in October that make Ofqual the regulator for all qualifications.  These were decisions taken by CCEA.  I am in the strange position of defending John O'Dowd on this one; his hands are not over this, but CCEA's are.

Mr Boyd: Chair, I think that we are talking at cross purposes.

The Chairperson: The principle applies.

Mr Boyd: The Minister made a decision on the continued offering of modular examinations in Northern Ireland.  CCEA will deliver the Minister's requirements on that.  The decision that CCEA made was in relation to the CCEA GCSEs taken in England.  That decision was not necessarily made on educational grounds but purely because continuing to deliver a small number of CCEA qualifications in England would require a significant investment.  We would not simply be offering the same qualifications that are offered in Northern Ireland.  Investment would be required, and that investment could not be justified by the returns that would come out of that investment.  That is a very straightforward and quite separate issue.

The Chairperson: Members, we are now out of quorum, so we cannot make any decisions, but we can still talk.

Mr Lunn: My questions are a bit more mundane than what you have been subject to.  I accept straight away that two examination errors and mistakes out of 422 is exceptionally good, but I am curious about how those two occurred.  What is the procedure for proofreading of examination papers?  Is that outsourced?  There are people who make quite a good living out of proofreading.  I have been told more than once that the only way to proofread something is to read it backwards.  Do the people who proofread understand the questions?  Are they expert enough to realise that a minus should have been a plus, which seems to have happened?

Ms Duffy: We have a very complex procedure in place.  The development of any one paper takes place over 18 months, believe it or not.  In that time, we have panels of principal examiners who write questions.  We have a separate and independent panel of revisers, under the directorship of a chair of examiners, who actually revise every single question to ensure that it is accessible and that it is the correct wording, etc.  They go through various revisions, backwards and forwards, for a period of time, until the chief examiner and the chair are both satisfied with the quality of the paper.

There is a qualifications evaluation committee meeting held, where the draft is finalised before it is sent off to the printer.  At the very end of the process, a scrutineer — a person who has had no previous involvement with any part of the process — sits the question paper on behalf of the candidate to ensure that all questions are answered, and also looks at the mark scheme to make sure that it has credited all credit-worthy answers that the scrutineer has come up with.  Thereafter, the paper goes to print and, at various stages of that process, it is proofed.  The proofing is done by the teams.  We do not have a separate range of proofers.  You are correct; I have often heard that it is to read it backwards as well.  So we have various eyes on the paper at various stages. 

The issue in relation to the printer error was very interesting.  The printer did, in fact, do what the examiners asked the printer to do.  However, the printer then chose to change a previously secure part of the paper in line with what he thought he had been asked to do for a particular subpart of a question.  They took it upon themselves to make a change in a previously secure part of the paper.  What that has taught us is that there is no part of a paper secure at all if it goes for a printer intervention at any stage.  We have brought it forward that every question paper that has a printer change will go through a thorough check again.  We are glad to report that we have no similar issues of printer errors in summer 2012.

Mr Lunn: It is pretty foolproof, then?

Ms Duffy: It is a thorough process, but it is amazing how we got two out of 422.

Mr Lunn: I do accept that.  I must tell you, as an aside, that, some years ago, we prepared an election leaflet for a Mrs Campbell and it was proofread by four of us, including me.  It was signed off to the printer by four of us, including Mrs Campbell.  The printer ran a dozen through, just to make sure that they were coming out right.  He then stopped the run and phoned me to ask whether there should be a "P" in Campbell, which there was not in the leaflet.  We had left out the "P", and so had Mrs Campbell. [Laughter.]

Mr Boyd: That is a good example of everybody seeing what they expected to see, rather than what was actually there.

Mr Lunn: Precisely.  I have one more question on the same theme.  I see that one of the things that you undertook to do was to work with other organisations to provide guidance to schools on the steps to take should an error be discovered during an examination.  What steps would they be?  If some pupil is brave enough to put his hand up and say that a certain question is illogical, what steps would be taken?

Ms Duffy: We have a number of feedback loops from schools during the process.  For example, we have a question paper comment log, which we hold, so any school or exam centre can contact us at the beginning, during or after an examination to say that there was a particular difficulty with a question, or that children coming out of the examination hall seemed distressed by a particular question.  That would then come to us and we would bring all of those comments to a senior examining panel before marking would start to see whether we should amend the mark scheme in any way.  Thereafter, there would also be a full review of any comments before awarding would take place to help us to set the grade boundary.

Mr Lunn: That is after the event.  What I am wondering is whether there is any mechanism, should a mistake be discovered during an examination, for instance if you had 50 pupils in a room all scratching their heads and saying that it does not make sense.

Ms Duffy: We operate a daily contact.  We have a single point of contact, which is made known to schools.  Schools can contact us the minute an examination paper is opened.  I will not discuss this summer's experience, but, in previous years, we have had issues in schools, when the school invigilator opened the wrong examination paper, gave it out to candidates, then realised two minutes into the examination that it was, in fact, the wrong paper and collected the paper.  In that situation, they would ring us for immediate advice and guidance.  In that case, we would give advice to bring forward the correct paper.  It is often the same exam, but there is a paper 1 and a paper 2 in each subject, and often they read the word "chemistry" and do not read that it is paper 1 or paper 2.We would then immediately send out individuals from CCEA to retrieve all the papers that had been given out and to safely secure the paper that should not have been opened.  We would then bring the papers that had been seen back to CCEA for a review panel to indicate whether it felt that a breach had occurred and whether we need to bring forward another paper in advance of the due date for the second paper.

Mr Lunn: I still wonder about a situation where, on the day, pupils are doing a three-hour examination and, after five minutes, one says that there is something illogical about a question.  For example, that a minus should be a plus.  The invigilator, who presumably has some expertise in the matter, can look at it and can agree.  Could he then make an announcement to the room to ignore the minus and say that it should be a plus, or is he bound not to interfere in the process?

Ms Duffy: We operate a process whereby, if an error is seen when a print run is already in a school, we can issue what is called an erratum.  We send a slip out to the invigilator, and ask that it is read in advance of children opening the examination paper to alert them to the fact that they need to make a change.  That change would be made at the start of the examination.

The Chairperson: Can I return to the workshop that was held in November 2011, following the decision on GCSEs.  The paper refers to "DE intervention" — I assume that was the Department, and its intervention was:

"Lack of evidence – to indicate that the linear approach is more robust".

The paper goes on to state that:

"CCEA Regulator to manage – important that CCEA Regulator works with counterparts in England and Wales on the comparability issue. Must ensure that students from here are not disadvantaged."

I know that those are two separate things, but, at the end of the day, they will come to the same point in time with how we go with our education system.  CCEA took a decision to do what it did with GCSEs.  What conversations were held with your English or Welsh counterparts in relation to what is now being debated by Michael Gove?  I know that Adrian said that the Minister is meeting his Welsh counterpart today.  That is welcome and is certainly good news.  What conversations has CCEA had, to ensure that, irrespective of who is doing this and what we are doing, we are ensuring that our pupils are not disadvantaged?  That goes back to my original point about the safety of children.  Have you had conversations with your counterparts in England, Scotland and Wales, and even the Republic, about that?

Mr Boyd: Yes.  Adrian will talk from an official perspective, but, from CCEA's perspective, we are in daily contact with colleagues in England and Wales on those issues.  Out of that daily contact, we build up a body of evidence, and, of course, we pass on detailed information to officials in the Department.  For example, we have talked daily to Ofqual about how it intends to deliver that which the Secretary of State for Education in England wishes it to deliver in the time frame that he has set out.  We do not believe that it is practically possible.  We feel that there is a whole series of practical issues that arise out of that and have communicated that to Ofqual.  The answer to your question is that we are in daily communication.

The Chairperson: Adrian, perhaps the timing of this is unfair, but it would be helpful for the Committee to have some indication.  You said that it would be a political decision in relation to what would happen.  What do you see as the time frame for that?  I am assuming that it is being discussed from a Welsh and Northern Ireland perspective as part of the discussions between the two Ministers.  Michael Gove's timeline has obviously been set out.  How are we assessing the implications of that?  What will the Department do now?  As with most of these things, I am sure that it is preparing papers on options A, B and C, and is perhaps going to consult on X, Y and Z, and all of that.

Mr Arbuthnot: The Department is considering the options that may be open to it.  The Minister's discussion with his Welsh counterpart is part of that process, which has been ongoing since we became aware of the issue last month.  I imagine that you will hear more from the Department and the Minister after the informal soundings, and we will then be scanning the horizon and looking at the implications.  You will hear more about that when you come back in September.

The Chairperson: OK.  Gavin, you will make enquiries about supplying papers to the Committee on the council's discussion about how it came to those decisions.  That would be helpful.

Mr Boyd: Yes.

The Chairperson: OK.  Thank you, Gavin, Anne Marie, Adrian and Richard.  You got off very lightly, Richard.

Mr Richard Hanna (Council for the Curriculum, Examinations and Assessment): I did.

The Chairperson: You did, but we are glad to see you back, and we wish you well.  Thank you all very much.