Official Report (Hansard)

Session: 2013/2014

Date: 26 February 2014

PDF version of this report (270.68 kb)

Committee for Education


Inquiry into the Education and Training Inspectorate and the School Improvement Process: Education and Training Inspectorate


The Chairperson: I welcome to the meeting Noelle Buick, chief inspector of the Education and Training Inspectorate (ETI), Faustina Graham, assistant chief inspector, Heather Jackson, managing inspector, and John Anderson, managing inspector.  We are pleased that you have all taken the time to come and be with us this morning.  Thank you for doing that.  We are in different surroundings today, but we are glad to be out of Parliament Buildings and in the partnership offices. 


Noelle, I want to express appreciation and thanks for all of the information that you have supplied to the Committee to date on the inquiry.  It might be useful, Peter, for members — I am sure that I speak on their behalf — to collate all of that information into one document, if that is at all possible.


The Committee Clerk: We will send members the link to all of the information that we have received in the inquiry.  It is all available through members' paper packs.


The Chairperson: If you can navigate your tablets, members, you will be able to access all of the information.  Noelle, thanks for coming with your staff.  I ask you to present, and then members will have questions.


Mrs Noelle Buick (Education and Training Inspectorate): Thank you.  A formal "Good morning" to everyone, and thank you for the opportunity to speak to you on 'Together Towards Improvement', self-evaluation and the process of inspection in primary and post-primary schools.  I will begin by briefly talking about 'Together Towards Improvement', and I will refer to it as TTI, which is our terminology.  I will also talk about self-evaluation, and then I will hand over to John and Heather, who will talk about primary and post-primary inspections. 


TTI is an ETI publication that was developed in 2002-03.  It may be used by schools to support self-evaluation.  It is not prescriptive that they use it, but it is also the framework for ETI inspections.  So TTI gives transparency to the inspection framework and promotes a common language for school evaluation and inspection, as well as a shared understanding of the factors related to school quality.  That is a strength that was identified by the Organisation for Economic Co-operation and Development (OECD).


Using TTI, inspection assesses the quality of education under three broad headings, five key questions and a range of quality indicators.  Those are the same broad headings, key questions and quality indicators that can be used by schools for self-evaluation. The three broad headings are leadership and management; quality of provision for learning; and achievement and standards. 


Under leadership and management, inspectors ask key question 1:  how effective are leadership and management in raising achievement and supporting learners? To do that, they look at seven quality indicators:  strategic leadership; action to promote improvement; staffing; accommodation and physical resources; links and partnerships; equality of opportunity, diversity and good relations; and public value.


To assess the quality of provision for learning, inspectors ask three key questions.  Key questions 2 to 4 are as follows:  how effective are teaching, learning and assessment; how well do learning experiences, programmes and activities meet the needs of learners and the wider community; and how well are learners cared for, guided and supported?  Each question has its own quality indicators.


The third and final area is the quality of achievement and standards.  Inspectors ask key question 5:  how well do learners develop and achieve?  The quality indicators are achievement, standards, progression and fulfilling potential.  Note that, as well as achievement and standards, we assess progression and fulfilling potential as part of that judgement on all inspections.


As the Chair said, members have a copy of TTI for post-primary schools.  Pages 12 and 13 outline the structure that I have just described, and, from page 14 onwards, you have the detail of the quality indicators to each of the key questions.  We are very happy to take any questions on those later.


In consultation with our stakeholders, TTI was revised in 2010 and is now more phase-specific.  We have a separate TTI for each of our phases, so there is one for primary, one for post-primary etc, and they use the language related to that phase, but the structure is as I outlined.  TTI is well used and respected as a self-evaluation tool, as well as being our framework for inspection.  In a survey of schools that were inspected in 2011-12, 90% reported that they found TTI quite useful or very useful, a statistic reported in the OECD report.  TTI complements and supports the Every School a Good School (ESAGS) policy; 'The Reflective Teacher', which was revised in 2012 to align with the General Teaching Council for Northern Ireland (GTCNI) competencies; and the school development planning process.  The school development planning circular was revised in 2010, which made self-evaluation more explicit as part of the school development planning process.


Self-evaluation is a process through which an individual teacher, groups of staff, the staff as a whole and senior management can reflect on their current practice to identify good practice and areas for improvement.  It should be a systematic process that also includes governors and, where necessary, members of the school community, such as parents and pupils.  The purpose of self-evaluation is to improve the experience of pupils, improve the quality of learning and teaching, raise the standards that pupils achieve and promote school effectiveness.


There is no statutory requirement for schools to carry out self-evaluation, but the school development plan regulations, which are statutory requirements, state that the identification of areas for improvement should be informed by schools' self-evaluation.  We evaluated school development planning as good or better in 75% of the primary schools that we inspected and 67% of the post-primary schools that we inspected.  We found self-evaluation to promote improvement good or better in 75% of the primary and 55% of the post-primary schools inspected.


As part of our mission to promote improvement in the interests of all learners, ETI works constantly to promote a culture of self-evaluation across the education, youth and training sectors.  However, we know that there is more work to be done on promoting self-evaluation, particularly in post-primary schools, as I have described.  As well as TTI, the self-evaluation process is supported by 'The Reflective Teacher', which helps teachers to evaluate their own teaching and learning; the "better" publications that you will have seen, such as 'Better English' and 'Better Mathematics'; the ETI inspection reports and thematic surveys; and our dissemination conferences as well as the district inspector work.  Self-evaluation and inspection are complementary, and the relationship between the two is central to stimulating improvement.  We are members of the Standing International Conference of Inspectorates (SICI).  It states, and I agree, that self-evaluation needs to operate in a framework of accountability, which encourages rigour and validates its authenticity.  So internal self-evaluation complemented by external evaluation ensures objectivity.  


To schools assessed as having an effective self-assessment process, a lighter touch of inspection can be applied:  for example, the sustaining improvement inspection in very good and outstanding schools that we are piloting.  This proportionate, risk-based approach ensures that resources are targeted where they are most needed and that good practice from good and outstanding schools is shared.


That is all that I have to say about TTI and self-evaluation.  I will now hand over to John, if that is OK with you, Chair, to talk about post-primary inspection.  Heather will then talk about primary inspection.


Mr John Anderson (Education and Training Inspectorate): Thank you, Chairman.  I will not go into too much detail on the way in which post-primary inspection operates, just sufficient detail, I hope, to allow you to hear how reflective practice and self-evaluation for improvement are part of that process.  I am happy to answer further questions.


Although people tend to focus on the inspection visit as an event in isolation, it is not; it sits within a long-term relationship between district inspectors and their schools.  District inspectors are already engaged with schools in supporting and providing a challenge to their school development planning and self-evaluation processes.  Therefore, they have long-term knowledge of their schools and insight into their local circumstances and the community with which they work.  Often, district inspectors see the annual revision of a school's development plan.  That plan is written for a three-year cycle but revised annually.  It is underpinned by action plans, monitoring, evaluation and reports on how well the action plans are progressing to achieve the school's current priorities.  More recently, they also have access to school data packs, which the Department's statistics team provides to the governors and principals of all schools annually.  The packs are updated throughout the year and the district inspectors have access to them as well.  So there is already a long-term engagement.


For a post-primary inspection visit, three days are spent gathering evidence in the school.  In advance of that visit, we ask the school to undertake a number of specific tasks to prepare for the inspection.  We reduced the extent of those tasks for the year just past.  Two years ago, we found that, typically, schools tended to write self-evaluation reports specifically for the purpose of the inspection and found that an onerous task.  That was an unintended outcome of giving schools advice that they had asked for.  Two or three years ago, self-evaluation was less common.  We gave them advice based on the headlines from 'Together Towards Improvement', which the chief inspector has just explained. Schools tended to take those headlines and write a report for the purpose of inspection, which, in itself, was not useful for us.   Now, we simply ask them to map or signpost for us their existing action plans and self-evaluation documents.  We request that they tell us who is responsible for these — whether there is an individual post of responsibility or a group or committee responsible — so that we know whom to talk to, what to read and what to follow up.  They do that in advance of the inspection.  That has reduced the perceived burden of preparing documentation.


Secondly, we ask the school to provide us with a subset of the data that it already holds in its school administration systems about the pupils, the curriculum and their progress.  It has been said during the inquiry that that is an onerous task.  People have mentioned large numbers of pages.  In fact, these are spreadsheets that are downloaded on to laptops and are not printed off.  I checked the amount of data that we extract from a school administration system, and it is less than 0·5% of the total data that is held by the school in any case.  It is a matter of downloading a very small proportion of that total, and it is done automatically through macros, so it is not a very difficult task.


Thirdly, we ask the school to benchmark and evaluate its own performance data on examination standards using the Department's circulars, which are published every year and provide benchmarking data for groups of schools.  We also ask the school to evaluate the wider skills and capabilities of pupils as they progress, develop and achieve their potential through the school.  You heard the chief inspector stress that that broader evidence on achievement of standards complements the specific look at the exam standards in the school.  When we ask the school to evaluate the standards of skills and dispositions, we are asking them to evaluate the extent to which pupils who have additional support with their learning are progressing and how pupils progress through their career in the post-primary school.  Data collection is a joint effort.  It is not a question of the inspectorate's gathering evidence on a school.  Together with a school, we can look at the evidence.  They evaluate it.  We go through a preparation day.  The reporting inspector who leads the team goes through a preparation day with the school a week in advance to go through the evidence and collectively confirm that it is an accurate reflection of the school's progress and performance.  So we jointly appraise and agree the picture of the school that emerges through that inspection visit. 


I mentioned that the inspection visit is over three days.  I will not go into great detail on how it is organised.  The important point is that we take the school's priorities in its school development plan as central to the inspection.  That is how we organise the inspection.  We are focused on, first, what priorities the school has identified for itself and, secondly, what its processes are to ensure that it achieves those priorities and can then move on to the next most important thing in the eyes of the school.  In doing that, we observe lessons.  We talk to pupils during and outside lessons.  We talk to teachers after lessons.  We look at the work of the pupils.  We examine the planning of the teachers individually and collectively as subject departments.  We look at their monitoring and self-evaluation processes, documents and reports.  We meet a whole range of post holders across the school and discuss their roles and responsibilities.  We meet the boards of governors.  Throughout all of that, we engage in professional dialogue with the school that is designed to achieve two things, the first of which is to evaluate the effectiveness of their own self-evaluation.  Also, we conduct that dialogue in a way that aims to contribute to building the capacity of the school to be even more effective in conducting its self-evaluation and development planning.


Self-evaluation, in the context of post-primary schools, flows from reflective practice, as set out in the competences by the General Teaching Council for an effective teacher.  I can recite those later if you wish, Chair.  Effectively, reflective practice is the day-to-day teaching and learning.  That is the work of improvement.  It is not an additional task.  In 'The Reflective Teacher',  page 17 of appendix 1 gives a sense of the flow of thinking that teachers undertake when thinking about their practice.  As they teach a lesson, they constantly ask themselves whether they are communicating effectively with their pupils and whether the learning from that lesson is as they intended.  What I am saying, Chairman, is that much of what a teacher does to reflect on their practice is about their professional judgement.  There are instances when they will use test assessment data to confirm that judgement, but, by and large, it is the professional skill of a teacher to be a reflective practitioner.  Teachers ask themselves the following questions:  am I helping or hindering the learning; at the end of the lesson, have I made a difference; do they know something that they did not know before; can they do something that they could not do before and, if so, how do I know that?  That is at the heart of reflective practice.

Self-evaluation is simply the aggregation of an individual teacher's reflective practice.  All those who teach the same subject will come together and discuss their reflection on their practice, and that comprises the subject department's self-evaluation.  When that is aggregated at the school level, you are then looking at self-evaluation for improvement at a whole-school level.  It is that kind of discussion that helps a school to set its priorities for the current year of its school development plan, which it then discusses with the wider community through the board of governors and in consultation with parents.  Schools vary in how they go about doing that.  It helps the school not only to identify those priorities but to turn them into the action plans that I mentioned earlier. 


The most effective action plans have a small number of priorities associated with success measures that are, from the teachers' perspective, measurable, as well as any  internal and external assessment or test results that they might have.  The reports that they produce are clear about the evaluation of the improvements that lead to the next cycle of improvement.  


Action plans that are less successful simply describe the tasks to be completed and monitor the tasks being done.  Any report produced tends to be descriptive rather than evaluative and is often lengthy, because it tells you all the things that they did but not whether they improved something as a result.  I can tell you all day what I do, but, if you ask me how effective I have been, that is a much harder question to answer.  Although this is not a very complex process, it requires intellectual thought to decide whether you are making a difference in what you do. 


In summary, Chair — I am happy to take further questions as you wish — we evaluate the appropriateness of the priorities that schools set in their school development plan and the effectiveness of their improvement processes in such a way that we intend will help to build the capacity of the school and its teachers and leaders to sustain improvement over the longer term.


The Chairperson: Thank you, John.


Mrs Buick: Chair, Heather will tell you a little about the primary process.


The Chairperson: OK.  Thank you.

Mrs Heather Jackson (Education and Training Inspectorate): Thank you, Chair.  As John outlined, common principles are applied across from post-primary to primary.  We are in the schools directorate, so there has to be a unity of purpose in school development, planning and self-evaluation.  So forgive me if I repeat certain concepts that John has already explored. 


I wish to talk to you about the primary-school inspection model and how we involve staff and the community in self-evaluation processes throughout the inspection.


Common to all inspection processes are the three principles of inclusiveness, transparency and accountability.  Inclusiveness is provided by seeking the views of the school through questionnaires for parents and staff and meetings with children, principals, coordinators and governors.  Throughout that process, we examine how well the children are developing their emotional and academic well-being. 


For transparency, all the documentation on our procedures is provided on our website for principals and governors prior to an inspection.  There is a 'Together Towards Improvement' document specifically for primary principals to access. 


Accountability is provided by sharing the findings in discussions with governors, children and coordinators, and comparing their evaluation with our evaluation and how that is mediated to the employing authorities.  We also involve associate assessors, who are practising principals, in our inspection process so that we have a current practitioner involved. 


The models that I want to talk to you about are the two- and three-day inspections.  The two-day inspection is aligned with smaller schools and lower-risk schools.  As John mentioned, this is about district information and a range of information from the Department.  We have a range of data to determine the risk-based approach.  We also have a three-day model. Regardless of which model is used, it is rigorous and robust, and similar and common procedures are used throughout. 


Common to all primary inspections is two weeks' prior notification of an inspection.  For the three-day model, there is a pre-inspection visit by the reporting inspector.  For the two-day model, there is a phone call to the school to set up the arrangements for the inspection.  The key objective of the phone call or the pre-inspection visit is to give the staff ownership of the inspection and to set out a framework and timetable for the various meetings but not the class visits, which are not predetermined.  It is about encouraging the school to organise its existing documentation.  For primary inspections, we do not expect any additional documentation to be generated. 


John talked about the school development plan required under the Department's 2010 regulations.  In primary-school inspections, that is the linchpin for the quality of evaluation, the priorities identified by the school, how well the school is delivering on those priorities, how successful the priorities are and how effective the action plans are in promoting improvement in children's learning.


Similar to the post-primary model, during the two- or three-day model of inspection, we observe at first hand the quality of learning and teaching in the classroom.  We regard each teacher as a leader in their profession and expect them to use their professional judgement.  As John said, they need to know how well the children are learning at the end of one lesson, a series of lessons and each term, so that is incremental. 


We discuss with children the quality of their maths, literacy and pastoral care.  We talk to groups of children and explore the extent of their knowledge of the statutory primary curriculum.  We examine the planning, monitoring and evaluation processes in the school's documents. 


We then meet a range of coordinators, and that is when we really get to the heart of where self-evaluation, both the process and the end product, takes place.  How did a school arrive at the priorities in its school development plan, what staff development has been worked on, and what has been the effectiveness of that improvement?  That is the core issue in self-evaluation:  how much difference do you make to the children and their learning?  Again, like John, we evaluate the appropriateness and effectiveness of the priorities. 


After an inspection, we report under the headings in 'Together Towards Improvement':  standards and achievements; quality of provision; and leadership and management.  On day two, we report back on the two-day inspection, and, on day five, we come back with a report to the school to discuss our findings with the employing authorities and school staff. 

In collecting evidence, we examine only what the school has, which could be a range of data.  The board of governors has a self-evaluation pro forma and receives the Department of Education's data pack, which we also have access to.  We discuss the findings and how well the staff know and understand the standards in the school.  If the school has internal standardised testing and that is made available, we review how well the children are achieving.  We use the internal data belonging to the school in our evaluation as well.  We also observe at first hand classroom teaching and learning.


The Chairperson: Before we proceed, someone's phone is interfering with the sound recording.  I am not guilty; mine is switched off. 


Thanks, Heather and John.  Faustina, you got off lightly, but we will come back to you in a minute or two. 


The recent OECD report stated:


"the setting of strategic or development planning requirements may not be adequate to stimulate an effective self-evaluation culture in all schools".


How has ETI helped schools to embed self-evaluation in light of that comment from OECD?


Mrs Buick: I will start off and then hand over to John if he wishes to add anything further.  If a school carries out self-evaluation without there being any external checks and balances, there is a danger that it may not be robust enough.  A school may have its own view of what is an appropriate standard, but you need that external benchmarking to determine the quality of self-evaluation, and I think that the inspectorate can provide that.  We provide, as John described in some detail, checks and balances on the robustness and rigour of the self-evaluation process.  Does it really identify what the key priorities are for the school?  Is the action planning that is put in place as a result of that appropriate and is it effecting improvement?  At the point of inspection, we give detailed feedback on the robustness of the self-evaluation process.


You also met our district inspectors, who play a key role in engaging professionally outside the formal inspection process.  If a school feels that it needs some support in undertaking the self-evaluation process, district inspectors provide that support.  The ETI's approach is a two-pronged one, through the district inspector work and the feedback at the time of the inspection.  Moreover, it devised the whole TTI as a framework to undertake self-evaluation.  It is not compulsory for schools to use the framework, but most schools do.


The Chairperson: Is that not more setting out the mechanics of it as opposed to determining that the methodology is right in the first place?  We all have varying views on OECD's wide range, but I think that the OECD document is going to be with us for a long time.  It is a very strategic piece of work, and we intend to do more work on some of the issues that emanate from it.  If you put strategic or development plans in place as a requirement — that ties into the point that John made earlier — the inspector will come along in conjunction with a school development plan, intrinsic to which are achievement, targets, goals and all those things.  However, if I have read the OECD document right, what it is saying is that that requirement in the framework:


"may not be adequate to stimulate an effective self-evaluation culture in all schools".


Are we at the point in the process at which there is a culture of self-evaluation embedded in our schools, and not because of the modalities and the way in which it works?  Is the methodology right in the first place?


Mrs Buick: The statistics that I quoted show that self-evaluation is better developed in primary schools than in post-primary schools.  We said that in 55% of post-primary schools self-evaluation was good or better.  One of the key points that we must make is that self-evaluation has to be owned by the school.  John described it as being bottom-up as well as top-down.  We cannot come in and say, "This is how you must do it.  These are your priorities".  Self-evaluation absolutely has to be owned by the school.  John, do you want to add to that?


Mr J Anderson: Chair, I completely understand the question that you are asking.  We come across it all the time in our discussions with principals about how effectively we can bring external evaluation alongside internal evaluation.  There is a Department regulation requiring a school development plan.  It has been in place since 2010 and has written throughout it the need to evaluate.  Schools can be tempted to regard that as something to comply with.  It requires them to have copious documentation about all sorts of policies, and quite rightly so.  Those are policies that they must have, such as a policy on child protection, to mention just one. However, that ends up with a filing cabinet full of documentation.  Schools comply with the requirement, but that in itself does not ensure a process of self-evaluation.


To be effective, self-evaluation at any time can focus only on a small number of priorities.  If you have more than two or three priorities, you do not really have any.  Therefore, the school at any one time has to consider what its most important priority is, what evidence tells it that that is the most important priority, and what the most effective actions are that it needs to undertake.


How do we support that?  That was the second part of your question.  For years, we have provided Together Towards Improvement, which is a framework for a process of self-evaluation.  We revised it in 2010 to have a version for each phase, as, prior to that, it was a single version.  When we relaunched TTI, we ran a summer school on the relaunch.  At any opportunity and if we can fit it into our schedules, which is difficult enough to do, we contribute to the staff development offered by organisations on self-evaluation leading to improvement.


I go back to my original point:  it is the continued dialogue between the district inspector and the school, not just during the inspection event but over time, that is the most effective contribution that we can make to helping schools think about how good their process of self-evaluation is.  Principals will say that they can produce plenty of documentation to comply with the Department's regulation, but that what they notice when we write reports about schools is that we report on how effective their process is, and the two are not the same thing.  We place our emphasis on the process.


The Chairperson: Is that still the case?  The OECD has said that the approach is not adequate and has questioned what else is not adequate.


Let me give you one example.  You rightly referred to 'Together Towards Improvement'.  The document identifies access to advice on STEM careers as being a quality indicator.  Why would it not identify access to the STEM curriculum in primary schools as being a quality indicator?  Would that not be more in keeping with giving validity and value to what we are about, rather than giving the perception that it applies to the circular?  It then becomes a very process-driven operation rather than something of value and worth to schools and the system.


Mr J Anderson: I will say two things on that.  I will come to the STEM question in a moment, but I will first address the point about how recently effective self-evaluation has been introduced in schools.  It has been the case only since the 2010 circular, which made it very explicit that evaluation has to be an important part of school development planning.  It was in previous circulars — I do not want to suggest that it was not — but it was made very explicit in 2010.


School development planning is a three-year cycle, so some schools were still working on school development plans that were developed under the previous regulations.  However, to be fair, in the past two years, we have seen the difference between the previous situation, in which not all schools were engaged in proper school development planning, action planning and self-evaluation, to the situation now, in which all schools do it.  Our evaluations are about how well schools do it, and that is often what you will find in our reports.  That is a fairly recent change, as far as my perception of planning in the post-primary sector is concerned.  Heather may have a view on the primary sector.


On the STEM issue, some of our Together Towards Improvement indicators concern curriculum provision.  That is a significant element.  One of our quality of provision questions is this:


"How well do the learning experiences, programmes and activities meet the needs of the learners and the wider community?"


One of the questions that we and schools explicitly address through self-evaluation is:


"Does the curriculum offer coherent broadly balanced programmes of learning which provide learners with clear progression opportunities?"


That is broken down into a range of indicators, and, of course, access to the STEM curriculum is part of the observation that we make of schools as well as the effectiveness of careers education in STEM subjects.

It is interesting that you mentioned that issue.  Not many years ago, we conducted a survey of the support for careers education in STEM particularly. Over a three-year period, our evaluation of careers education has moved from being effective in around 30% of schools to over 80%.  That is because of the existence of the Preparing for Success policy, which is a joint DE and DEL policy; because of the intervention of the Curriculum Advisory and Support Service (CASS), which ran a specific project on promoting effective careers education in STEM; and because we inspected it and reported on in it in every report of every school.  That is an example of levering a significant improvement when you bring together an effective policy, effective support and effective inspection.


The Chairperson: I am trying to stay within the parameters that I set myself and the Committee earlier by staying on the issue of self-evaluation.  That is crucial if we are to get a perspective on the issue.  The question that that poses is why there is a huge issue with science in our classrooms, particularly in primary schools.  Perhaps Heather can try to give us some indication as to why that is, but, that is certainly the sense that the Committee has following our visit to the BT Young Scientist and Technology Exhibition and based on recent correspondence from the Department.


This may a very strong way in which to put it, but it was very regrettable that, at the this year's exhibition, there was not one school from Northern Ireland represented in the primary section.  Why was that the case?  When we started to make enquiries, we discovered it is because the area of learning known as the World Around Us is very generic, and science as a subject is no longer being progressed and developed.  Is that not the sort of thing that should be picked up so that there is a continuum from primary to post-primary?  On the very specific issue of teaching science in our classrooms, if the self-evaluation process is the robust mechanism that we are told it is, it should have been picked up as a problem.  We have to identify how we can address it in conjunction with the schools and through the curriculum in order not to have a situation in which science is picked up in the post-primary sector, at which stage its interest, value and worth will have been lost to a certain number of our young people.


Mrs Buick: Heather will come in with some of the detail about the World Around Us in primary schools, but we are —


The Chairperson: Not that we are cynics, but we are glad that we raise these issues, because we know that the ETI has undertaken an evaluation of the World Around Us.  We are glad that that we may have prompted that review.


Mrs Buick: Chairman, that is one of the very points that I wanted to make.  We plan our schedule of work very well in advance, and our study of the World Around Us, specifically in primary schools, was well in place before this inquiry even started.  I hate to disappoint you.


The Chairperson: Noelle, you have just disappointed our importance.  That is a bubble burst.


Mrs Buick: At least we are on the same page in absolutely accepting that it is an area that needs further scrutiny.  As you know, the World Around Us is part of a wider range of subjects that includes history, geography and science.  I suppose that I would say that the Department has invested quite a lot of money in training teachers to enable them to deliver STEM subjects and in providing resources to enable the delivery of STEM.  I think that there is a strong focus on the STEM agenda.


We are not yet in a position to be able to report on the findings of the review of the World Around Us, as the survey is being undertaken at the moment.  However, I am sure that Heather can give you an outline of some of the things that we see in primary schools around the delivery of science.


Mrs Jackson: I endorse everything that Noelle has said.  The survey is ongoing, and it would be premature to reveal any findings, as we have not got all our evidence together as yet.


With the change in the Northern Ireland curriculum in 2007, science, technology, history and geography were put under the one heading of "the World Around Us".  There is no differentiation in the quality or extent.  They are three contributory subjects.


Schools are addressing the issue of entitlement to the curriculum, and, in the survey, we are finding that there is variation in practice.  Some of it tends to relate back to staff confidence and development and to access to CASS training in science and technology specifically.  Teachers can and do access scholarships and bursaries to the National Science Learning Centre in York, and a STEM officer linked with the Belfast board facilitates a range of courses throughout Northern Ireland to train teachers, as do certain board officers.


The Chairperson: I have one final point to raise before I hand over to members.


John, in your post-primary domain, 45% of schools have an evaluation that is not satisfactory.  The ETI is saying that self-evaluation is embedded as a process and part of the overall machinery of schools.  From your experience, where do you think that our teachers, who are the key drivers in our education delivery, are at personally with the concept of self-evaluation?


Mr J Anderson: As you pointed out, we have evaluated that there are more schools that are less than effective at post-primary level than there are at primary level.  You are asking whether that is down to teachers' individual effectiveness in their own reflective practice, and that has to build on how well the GTCNI competences form part of their working day, as they teach lessons and have conversations with their colleagues who teach the same subjects.  Therefore, the question has to be this:  how well do the GTCNI teaching competences pervade the ongoing staff development processes, including performance review and staff development (PRSD), in schools?  I believe that that is an open question.  We need to know how much that is the case, and, obviously, we are asking that question because we are not convinced that the staff development processes can inform ongoing PRSD effectively enough.


The Chairperson: Should it be mandatory?


Mr J Anderson: Should what be mandatory?


The Chairperson: Self-evaluation.  Although it is in regulation, it is not a statutory requirement.


Mr J Anderson: It is there as part of the Department's regulations.  It is also there because it is part of the competences of being a teacher.  How well does staff development operate in schools to build that capacity?  We have told you how we contribute to it, but we need to ask how effectively it is working in schools through all the staff development operations, including PRSD.  Of course, there are limits to our access, because it is related to promotion, and so on.


Mr Lunn: Thank you for your presentation.  I do not know how I will ever stay within the parameters that the Chairman set out.


The Chairperson: I did reasonably well.


Mr Lunn: Forgive me and pull me up.


It seems like a well-structured system.  As an outsider looking in, it certainly appears to be well structured.  You can see a pathway through it, and so on.  How does it compare with what happens in the rest of the UK?  Is the procedure much the same there?


Mrs Buick: Do you mean the inspection process or self-evaluation?


Mr Lunn: Self-evaluation.


Mrs Buick: It is very similar.  Self-evaluation has been around in the English system, with which I am familiar, for quite a long time, going back to the 1980s.  The process is identical.  It is about a school or a provider owning, as John described, the reflective practice in its organisation and determining what the strengths of its work are and what areas it needs to improve.  There are certain barriers to self-evaluation.  There needs to be a culture of openness, and there has to be a willingness to be absolutely rigorous in your self-assessment of what you do well and what you can improve.  You need to be absolutely honest in how you compare yourself against other schools and providers that are providing a similar service in similar circumstances with similar groups of pupils.  You have to have good consultation with and ownership by members of your staff.  It is not something that is done in an isolated room by one person.  You have to have the groups of people that we talked about involved, and you also have to have a really wide evidence base of all the work that you undertake.  Unless you have those things in place, you will have a self-evaluation process that is not as effective as it could be, but the process and the impact are very similar.


Mr Lunn: Is it a stressful process for teachers and principals?


Mrs Buick: Reflecting on how well you are meeting the needs of children and learners in your care and for whom you are providing the best education that you can should be part and parcel of your everyday work as a teacher and principal.  I do not think that it is stressful.  It is part and parcel of the work that schools do.


Mr Lunn: I understand that self-evaluation has to be rigorous in order to have any meaning and to encourage or almost force people to be realistic and revealing about themselves.  However, you would not believe how much difference there is between what you have just said and what we hear from teachers, who seem to regard it as extremely stressful in some circumstances.


Mrs Buick: The self-evaluation process?


Mr Lunn: Yes.


Mrs Buick: I am not sure that that is the feedback that we get about self-evaluation.


Mr J Anderson: "Stressful" is not the word that I would use.  Teachers will say that it is burdensome or time-consuming, because self-evaluation can be made more complicated than it needs to be.  We see that occurring, but it is not stressful, because it is not outside teachers' control.  It is their own process.  It is not like an inspection, where a visitor gives you an objective, external view, which they may find creates anxiety for a while until they get used to it.  However, it is not stressful in that sense.


The point about effective self-evaluation is that, if it is made too complicated, it is less effective.  I go right back to the definition of "reflective practice", which is that the day-to-day work of learning is the business of self-evaluation.  You are thinking all the time as a teacher, "How well is my teaching going?  Am I achieving what I intended to achieve?"


You then need to take time to talk to your colleagues and be given proper staff development time in school to work with them in your subject area or, in the case of primary education, your Key Stage.  You then have the opportunity to bring that up at school development level, which is an important task that needs to be done as efficiently and effectively as possible.  That is when self-evaluation works well, instead of it becoming a bureaucratic process in the school.  However, I would not have recognised it as being stressful.


Mr Lunn: You place great stress —  sorry, there is that word again — on the starting point of the school development plan.  It may have been you, John, who said that three or four priorities would probably be enough in most circumstances.  Do you not find that school development plans are broadly similar and that the main priorities would normally be the same, or do you get the odd one where there is a clear difference?


Mrs Buick: It should reflect the priorities of the school, and every school is different.  They are at different stages on the improvement journey.  Their intake of pupils may be different, so it should be owned by and be reflective of the school.


Mr J Anderson: I think that the chief has put her finger on it.  Schools are at different stages of improvement, so, yes, the priorities do differ.  You tend to get themes that are reflective of government priorities, so it not surprising to find aspects of literacy and numeracy occurring, but the aspect will depend on the pupils, teachers and the nature of the practice of teaching and learning in that school.  The issue for the school is in identifying the aspect of it.


If teachers agree that they feel that the pupils in their school are less confident verbally and orally in discussion than the teachers think they should be, that may become a priority for one or two years in the school development plan, whereby teachers act in an organised and coherent way through all their subject teaching to try to improve pupils' confidence to speak out in answering questions, to make presentations to one another and to engage in group discussion.  When it is done effectively, they will monitor and track their efforts and evaluate at the end of the period whether those efforts to bring about an improvement have made a difference.  If it has made a sufficient difference, they may move on to the next issue, which is to extend pupils' writing or their ability to use numbers more effectively outside mathematics, science and technology.


You do tend to get themes, but they will differ according to the school.  If the school knows itself well, it will be precise in identifying what it needs to be doing over the next year or two years.


The Chairperson: It will be different the next year and the following year.


Mr J Anderson: It could be.


The Chairperson: Your cohort of pupils passes through your school on a cycle, so the challenges that teachers meet will vary and be different.


Mr J Anderson: Yes.


The Chairperson: I suppose that the issue is how teachers respond to that, and how the management of the school addresses that, which is perhaps the more important issue.


Mr Lunn: You mentioned associate assessors.  In simple terms, what is their role in the process?


Mrs Buick: In the inspection process?


Mr Lunn: In whatever part of the process you care to identify.  I just wonder what they do and how seriously their input is regarded in the process.


Mrs Buick: I will come to Heather in a moment, but I think that you met the associate assessors.  I have not yet heard how that went, but there is no doubt that associate assessors bring the current experience of the sector to inspection.  In fact, they are in a very good place to look at self-evaluation because they have been involved in the internal self-evaluation of their own school.  When they join the inspection they are also involved in the evaluation of the effectiveness of self-evaluation in the school that we are inspecting.  They bring great currency to the inspection process and complement the skills of the full-time inspectors who are experienced professional evaluators.  The Organisation for Economic Co-operation and Development (OECD) recognised the value of the associate assessors in our inspections complementing the experience of our experienced inspectors, so much so that they put it in the good practice report and in 'Synergies for Better Learning'.  They bring a positive contribution to inspection.


Mrs Jackson: The associate assessors are in primary schools with us on inspection teams.  They go through a rigorous training session of observing lessons, how we conduct an inspection and the quality indicators that we are looking for.  They are trained how to evaluate before they go into schools.  That is essential.  They also have specialisms that we use on inspection.  While observing in classes, they will do evaluations and be with us in literacy coordinator interviews.  They will be aware of current initiatives, be able to explore aspects with us during a joint interview with a coordinator or a senior leadership team, and they will bring a current view on education to us.  We have the overall benchmarking, but they have specific specialisms that they bring to the team.


Mr Lunn: I had not heard of associate assessors until a few weeks ago.  That is my starting point.  I am a complete ignoramus on this matter, but it sounds like a very good system.  It is like a lay assessor.  It compares with other situations, as long as their input is valued and they have some control or responsibility in the overall outcome.  As you said, Noelle, you have not seen the results of last week's discussions yet.  You might want to reflect on those when you see them because I do not think that they feel as valued as you seem to value them.  I wonder what happens when they disagree with the outcome.


Mrs Buick: I will ask Faustina to speak in a moment because she might want to contribute to this.  I find that astonishing.  As an organisation, I feel that we absolutely value associate assessors.  We have 197 of them.  We recruited a new tranche in the summer and had more than 200 applications for 80 to 90 places.  If we need to reinforce the message to associate assessors that they are highly valued, we will do that.  We are meeting them over the next couple of weeks for training with them.  I am really surprised by that comment.  Faustina, would you like to come in?


Mrs Faustina Graham (Education and Training Inspectorate): If that comment has been made, we have to accept it and address it.  It is unfortunate if that is the case because it is absolutely not our intention in working with our associate assessors.  We stress to associate assessors that they are full members of the inspection team, and they contribute to all the discussions.  There are absolutely no discussions that I can think of from which they would be absent.  I have to say that internally in our organisation we disagree too.  In an inspection team, there will be perspectives around the evidence that you have seen across the three days that you have been there.  The purpose of our moderation meeting on a Thursday, in most instances in the schools, is for all the evidence to be collated.  In that situation, we each perform a challenge function for one another with regard to the evidence that we have collected.  Some discussions can be quite robust, and people will have certain perspectives.  However, all that is geared towards reaching, as we see it, an agreed conclusion at the end of an inspection.  Certainly, perhaps, people who have been on only a small number of inspections are still growing and developing discipline.  When we present evidence, what we have to think about are the arguments and issues that we are dealing with, and there is nothing personal in any of that.  That is something that we all develop over time; we appreciate that we are debating issues, not personalities.  However, we are willing to take on board any comments from the associate assessors that suggest otherwise, and we will do something about them.


Mr Lunn: I am nearly finished.  We are not talking about an exact science.  You will never get unanimity in evaluation and judgement but, hopefully, you can achieve an agreed verdict.  All I said was that when you see the responses of the associate assessors you may want to reflect on them.  You can take it that I would not have said that had I thought that it was totally in agreement with what you have said.  I will leave it at that.  Thank you very much.


The Chairperson: On that point, Noelle, after this meeting we will forward to you a copy of the informal briefing that was held with the associate assessors, as we have done previously with inspectors.  None of the comments is attributable, but they give a sense of what came out of the event.


Mr Newton: I thank the delegation for coming along.  It is obvious that there is a passion in the group, and that is particularly reflected by some of the comments that John made on self-evaluation.  Is it not all a bit woolly at the end of the day when you go into a school and come out with a self-evaluation report that is not set against a common or national standard of self-evaluation?


Mrs Buick: I will start off and John can join in.  I would never describe inspection as woolly.  'Together Towards Improvement' is the framework that we use for inspection.  As it says in the OECD report, it is open and transparent.  Everybody can see how we make our evaluations.  It promotes a common language so that we are all speaking to it.  We have a framework for inspection within which there is a clear structure that John and Heather outlined as to how we undertake inspection activity.  We look at data benchmarked against similar schools; we look at a school's internal data; we use first-hand evidence of the teaching and learning and the work in pupils' books; and we talk to co-ordinators, governors and pupils.  We collect a wide range of evidence.  All inspectors are professional evaluators; that is what they do.  They are in classrooms every day that a school is open, so they have a clear picture of what are acceptable standards — good as well as above and below good — for making that decision where it is needed.  We have a clear framework, a clear structure for how we undertake our inspections, and good, professional inspectors to undertake that work.  That gives it a clarity of purpose and understanding.  Faustina explained the moderation and quality-assurance processes.  The judgement that we come out with at the end of an inspection is a robust one based on solid first-hand evidence.


Mr J Anderson: I have nothing to add to that.  Sorry; other than, perhaps, exactly as the chief says, on this particular aspect:  when a school looks at its performance in public exams against benchmarking, "woolly" is the last word to describe that process.  That is only one part of the evidence base; the rest is, as the chief said, part of a professional dialogue with schools by people experienced in schools and in inspection.


Mr Newton: The words "inclusiveness", "transparency", "accountability", "rigorous", "robust", "common language" and "systematic" are used, but nobody talks about measurable.


Mr J Anderson: Absolutely, we are talking about measurable; effective self-evaluation has measurable success indicators.  Where there is a lack of such measurable success indicators, that is where you get self-evaluation that is simply a catalogue of tasks that have to be done and monitoring that simply checks whether those tasks have been done.  The reports are long descriptions of what people did without any attempt to evaluate, in a measurable way, what the learning gains were as a result. 


What often compounds the weakness of a less effective self-evaluation is that no one is ever clear from the outset what the intended learning improvements would be.  Let me give you a well-known example:  schools will invest thousands of pounds; it used to be on interactive whiteboards and now it is on iPads, and the objective is to purchase iPads.  The first question is where the objective is that clarifies what learning improvements would result from using a particular technology tool and the measurable indicators to show how teachers and pupils used that technology tool to raise standards.  That would be an effective evaluation; without such measurable indicators it would be ineffective.  That is the kind of evaluation that we undertake when we look at whether self-evaluation in a school is good.


Mr Newton: The quality indicators in those areas are management and leadership, which has four or five subdivisions; quality of provision of learning, which also has subdivisions; learning experiences and care and guidance, which are all subdivided; and achievements and standards.  What is the scoring system?


Mrs Buick: I think that you submitted that question to us.  There is no scoring system; we do not attach weights and measures to each key question.  We assess a school in the round using the five key questions and take into account the context of a school, but we do not attribute a score to each key question and come up with an aggregated average, which was, I think, how some people implied we carried out an inspection.


Mr Newton: How would you specifically measure any of those areas?


Mrs Buick: Through an evaluative process.  I will ask John and Heather to come in on that, as they do that all day, every day.  We know that the key factor in young people's success in school is the quality of teaching and learning.  We spend a great deal of time on inspection looking at the quality of teaching and learning, and we would expect it to be good or better in order to give pupils the best possible life chances. 


We also know that you find the most effective teaching and learning where you have good leadership and management.  We focus on the strengths of leadership and management, certainly around self-evaluation and determining priorities for the school, and you would expect that to be reflected in the outcomes for the learners.  You would expect those to be good or better if you had good or better teaching and learning and good or better leadership and management.  There is an evaluative process:  it is a whole school evaluation using the whole of the Together Towards Improvement framework, not just specific aspects of it.


Mr J Anderson: There are seven questions that we ask under the heading of leadership and management.  I recently gave you an example of how we would evaluate action to promote improvement; that is clear-cut.  You can see whether it is effective because a school can tell you that it knows how it has made a difference.  That will give you great conviction that it has the capacity to improve through its own internal ability to self-evaluate, plan for and bring about improvement.  Another key indicator under that heading is the effectiveness of the strategic leadership.  How do we evaluate that?  By using questionnaires to get staff views on communication in a school and the effectiveness of its leadership.  During the three days, we conduct interviews with post holders, with the principal and the senior leadership team as a group and as individuals, with those in middle management who co-ordinate aspects of the school's provision, and with heads of departments.  We ask questions about whether there is a common and understood moral purpose in the school and how well that is articulated by everyone we talk to.


When I talk to a principal and hear that there is a clear understanding of the vision and mission of the school and when I talk to someone who has just recently been appointed as the literacy coordinator and I hear the same messages and the same language, and when that is repeated over three days when the members of my team conduct those interviews, that provides a solid evidence base that a school has effective strategic leadership.  If those characteristics are missing, that is significant evidence that the school is poorly led with a lack of common purpose or understanding of what it is doing, and with people engaged in the administration of tasks rather than understanding how they can raise standards.


It is in the conversations during inspections, which Faustina mentioned earlier, and on the final day, on the Thursday, when we conduct the final moderation meeting in which the associate assessors are fully engaged as members of the team, that we hammer out our individual views about everything that we have heard and read.  That gives us a sound evidence base that we can stand over when we come to report on the Friday, which is the final day.


If a school does not recognise itself in the messages that we are giving it, we have to be clear that we have an evidence base.  It may not recognise itself because it has a distorted view of how successful it is and sometimes the messages are not welcome.  More often than not, however, a school will agree that it is a recognisable evaluation of the school as one that it knows.  Where there are challenges, a school will act on them even if it is not happy, and, over time, it adjusts to the fact that we were giving it an accurate story and that it needs to move forward.


Mr Newton: Do you move outside the direct teaching staff to talk to administrative and support staff?


Mr J Anderson: We look at how classroom assistants work, and we take account of how, for example, ancillary staff who support teachers, such as science technicians, play a role if it is important or significant, but not the administrative staff or the catering staff.


Mrs Buick: Those staff can feed their views in through the staff questionnaire, so we take their views on board.


Mr J Anderson: They do, but we do not interview them.


Mr Newton: I cannot remember who used the word "culture", but the culture of a school is very important.  If you want the culture of a school to be learning-driven, how can you achieve that if you do not involve all the school staff in the exercise?


Mrs Buick: We involve most of the school staff — those who are involved in teaching and learning and who are directly involved in ensuring the best educational outcomes for the young people.  We take the views of the support staff through the questionnaires, so they make a valuable contribution to the running of a school.  We do not in any way leave them out, but our focus is on the quality of the educational experience for young people.


Mr J Anderson: We do not interview office or cleaning staff, for example, but we look to get a sense of how a school saw itself as a whole community.


Mr Newton: That is what I am trying to get at.


Mr J Anderson: Exactly.  For example, when a school undertakes staff development, does it limit that to teaching staff only, or does it involve the science and ICT technicians, the art assistants and the learning support staff who work with children with special needs?  When a school sets up its school development plans and identifies its priorities, does it consult only the teaching staff, or does it consult the whole school community and the parents beyond that?  Those are questions that we ask to get a sense of a school as a community.


Mr Newton: Someone asked earlier about the training and qualifications of those who carry out self-evaluation in a school.  Is there a continuous professional development (CPD) programme for those involved in it?


Mr J Anderson: It is not uncommon now to find that part of a school's development plan contains a staff development plan as well.  You asked whether it was similar in every school.  One of the things that is quite similar is that many schools now provide staff development to help staff to understand how to make use of exam and assessment data.


Mr Newton: Is it measurable?  What you mean by "it is not uncommon"?


Mr J Anderson: By that I mean, "I am finding more commonly now."  Yes? [Laughter.] Let me finish the point; I hope that I can make it clear.  Because teachers now realise that there is assessment data available to them on the school's computer administration systems, staff development priorities in the schools are often — perhaps my colleagues can help me here — focused on helping teachers to understand how they can use assessment data themselves as part of the evidence that they use to decide whether they are being effective in their teaching in a way that will adjust their teaching to be more suitable to the needs of individuals in the class.  You asked earlier whether there are common themes, and yes, that would be a common theme.


Mrs Buick: If you want to quantify it, 78% of the primary schools that we inspected in the last chief inspector's report were "good or better", and so were 65% of post-primaries.  That is the kind of activity that we would expect to find in "good or better" schools.


Mr J Anderson: The schools heard that through the chief inspector's report; they read the reports and respond to them.


Mr Newton: Would you ever think about benchmarking your process against a system such as Investors in People?


Mrs Buick: The Department undertakes an Investors in People assessment, of which we are part, in which our staff development came out incredibly strong.  However, we have our own benchmarking activity in that we undertake Customer Service Excellence (CSE), and we have just been through the reassessment of it.  That is not unlike Investors in People in format, except that it focuses on our customers.  We also have the Northern Ireland Statistics and Research Agency (NISRA) independent survey of those that we inspect, and that enables us to determine the quality of our inspection process.  As I said, we are members of the Standing International Conference of Inspectorates (SICI), which benchmarks our performance.  The OECD compared our performance with 32 other countries, and we came out very positively from that.  So I think that we do —


Mr Newton: I am interested in self-evaluation in a school and whether it is benchmarked against a national standard.


Mrs Buick: Do we self-assess?  We undertook a self-assessment exercise as part of our corporate plan in March 2013, and we looked at the things that we do well and at those that we need to improve.  The things that we need to improve are in our corporate plan; therefore we do undertake self-assessment.


Mr Sheehan: John, you said that sometimes reports from schools are descriptive rather than evaluative.  Let me labour Robin's point about training.  Are teachers trained to self-evaluate?  Who trains them?  If they are giving descriptive reports, is something lacking in their training?


Mr J Anderson: Yes.  I will not repeat what I have said already about our contribution.  However, the main source of training for improving schools' self-evaluation comes from two places:  one is the advisory service in each board; the other is the regional training unit, which has officers who work in that area as well.  They run courses and provide support, particularly, for example, for those in middle management in school.  Part of that process is dealing with improving self-evaluation.


Mr Sheehan: Is there correlation between the reports that are lacking in evaluation and poor inspection outcomes?


Mr J Anderson: Yes.  If a school is not effective in its self-evaluation, that is a reflection on the school.  Therefore in an inspection outcome we try to evaluate accurately and reflect on where a school is.  We often use the language of an "improvement journey".  Where is a school in its improvement journey?  Is it fairly immature, working descriptively and not very evaluatively and not having success measures?  Has it become effective, and has it considerable capacity in that area to the point where it can sustain its own improvement without external help?  There are schools all along that scale, and those that are less effective need the input from CASS and from the regional training unit.  In fact, those that are evaluated as "less than satisfactory" get support through formal intervention in a structured and organised manner.


Mr Sheehan: I want to explore the connection between self-evaluation and the external inspection and the sequencing of that. For example, is the district inspector in any way involved in the self-evaluation process?  When the external inspectors go in to begin the inspection, is the self-evaluation report the first port of call?  Do they look at that first and then carry out their inspection after that?


Mrs Buick: I will maybe ask Heather to come in in a moment.  If a school feels that it needs some challenge, support and advice around the self-evaluation process, it is part of the role of the district inspector to help it with that.  Also, the school development plan, which includes self-evaluation, is absolutely at the centre of the inspection process.  As John described, in post-primary schools, the inspection is tailored around the priorities that are identified in the school development plan, so it is a really important part of the inspection process.  Heather, is there anything that you want to add?


Mrs Jackson: The school development plan is very important in a primary school because it is about pacing development and baselining what the school knows.  For example, it could be looking at improvement in reading scores and reading standards.  It is about it baselining its staff development.  Do they have the capability within the school or do they need to get external support to develop the teachers in teaching reading?  Have they tracked the journey of the children?  When they do make an amendment or an innovation, is there an incremental development to see whether it is making a difference?  Is every member of staff on board? 


It is about providing support for those who need additional support, so it is really the step-by-step journey of making improvements in a school, for example, in the teaching of reading.  Do they know at the end of term, with two terms a year over three years, that they have made a significant difference to the quality of the learning of those individual children?  We have that very important challenge function to say, "This is what you identified as a whole school and agreed in primary school".  It has to be a process of agreement.  Then we need to look at whether the terms and conditions as regards a timescale of implementation have been agreed and whether there has been an immediate impact on the learners out of that.


The external evaluation is through a district visit.  The district inspector will visit the school by request or they will request a meeting with the school.  In that case, there is usually a comparison of the quality of learning and teaching in the classroom.  In some cases — it is a more recent innovation — we are doing paired visits with the principal by agreement with the teachers and teachers' union.  They observe and we quality-assure the internal evaluation, be it with the principal or with the coordinator — the literacy coordinator, for example — to see the journey that the school is on and to give indicators regarding further progress. 


The school development plan and the use of the district inspector is an organic process.  On inspection, we do look at that.  That is what the principal talks about at the very beginning of an inspection, by saying, for example, "These are our areas for development.  We have arrived at these.  This is where we are".  We encourage the school, through the school development plan, to benchmark where it is in its journey.  We evaluate the quality of learning and teaching throughout the school.  We then come back, as John referred to. 


Mr Lunn, you mentioned stress.  Sometimes, a badly planned or badly paced school development plan causes stress internally.  It is not caused by the inspectorate doing an inspection.  That is where the inspectorate team or the district inspector can intervene to say, "You have too many priorities.  The initiative is not being embedded well enough and you need to stop, re-evaluate where you are and address the issue in a more coherent fashion".


Mr Sheehan: Finally, on the issue of principals undertaking classroom observation, TTI suggests that classroom observation should be one of the quality indicators, but PISA 2012 tells us that only 12% of principals were undertaking classroom observation.  Do we know why that is?


Mrs Buick: I think it is a landscape that needs clarification with the teaching unions.  I know that we have some representatives here today.  In many schools, we now see staff undertaking lesson observations within the school.  Certainly, in a considerable number of schools, we see a lot of peer observation, with staff observing each other and picking up on good practice.  However, I think that there is no better way of sharing good practice in a school than to have an open-door policy and enabling all staff — senior staff and teaching staff — to observe each other.  That is definitely the way forward in improving the quality of teaching and learning, but there are some landscape issues to be ironed out before that is the case in all schools.  We are certainly moving in the right direction.


Mr Sheehan: "Landscape" is a euphemism for reluctance to allow that to happen.  Is that what you are saying?


Mrs Buick: There are some difficulties in some schools around that process, which I think need ironing out.


Mrs Graham: With regard to the whole issue — we used the word "culture" earlier — of classroom observation, it is quite a confused landscape in the sense that classroom observation can be used for all sorts of purposes.  If you are not completely clear about why you are embarking on classroom observation, it can seem to be only for accountability purposes as opposed to effecting improvement in the work that is happening in the classroom.  Some of the work that we are exploring at the moment with the associate assessors is around the whole concept of how to evaluate learning effectively as opposed to seeing teaching as being about a performance or the individual and how someone does particular things. 


In conjunction with our associate assessors, what we are trying to explore, and we will be working on this further next week, is how all of us can place the emphasis on evaluation of what is happening in a classroom on the impact on what children actually learn at that point in time.  So, it is not about the individual and being good, bad or indifferent; it is about the actions and pedagogy that are used in the classroom and how that impacts on children's learning. 


We have been honest enough with our associate assessors to say that we do not have all the answers to those questions.  For all of us, it is a joint piece of work to try to remove the idea that, particularly for people in the teaching profession, if you are not perfect, you are awful.  That is a culture that we need to change.  I can say that having experienced it myself as a teacher.  If someone says something that appears to be critical, you can take it personally as opposed to thinking about how it impacts on the quality of learning.  As a profession, that is something that we need to move ahead on so that all of us work collaboratively on how we improve children's learning as opposed to focusing on ourselves and whether it is some reflection on us that a particular lesson went well or did not go well in that situation. 


Our mission statement comes back to being about promoting improvement in the interests of all learners, not promoting self-evaluation.  Self-evaluation without improvement is just an action that does not result in anything.  I think that the same is true of all of the work that we do.  It is more about trying to ensure that there is improvement, first and foremost, than just the evaluation at that particular point in time of where a school is at.  So, what we really seek to do is try to point the way forward for schools, rather than simply saying that it is an overall summation or a measurable outcome of where they are at that moment.  We absolutely try to do that, but, first and foremost, we want to help schools to improve their provision for young people.


The Chairperson: Obviously, this is a key issue.  There is 60% of teaching observation in England and 12% in Northern Ireland.  Whatever way we try to describe it as "landscape", the reality for us is that there is a problem and a difference of opinion between staff and the system about how that should all be put in place. 


In our packs, members have a copy of an inspection report with the school's name deleted.  I accept that it is from October 2009, so things may have changed.  If you sit down and read that report, you see comments, like the one on page 63, under the heading, "Quality of Learning and Teaching", which state:


"In the best practice observed there were opportunities for the pupils to engage actively in their learning."


It then stated:


"However, in almost half of the teaching observed, there was less effective practice.  This was characterised by lack of pace and challenge, reflecting the teachers' low expectations of the pupils, with insufficient focus on the intended outcomes."


How do you square that circle with the comment that was then made about the questionnaires?  There was a whole debate about the questionnaires.  Some teachers believe that they just set the teachers up for unfair criticism.  The report states:


"Five support staff completed a confidential questionnaire.  A majority of the staff who responded raised concerns relating to aspects of leadership and management and communication within the school."


So, where did the problem lie?  Did the problem lie with the individual teacher who had, according to the report, low expectations of their pupils and whose teaching was characterised by lack of pace and challenge or did the problem lie with the senior management team of the school, which did not have a process of staff evaluation and communication, leading to the poor outcomes in the classroom?  Does that not all go back to the problem of the landscape of the classroom or whatever way you want to describe it?  How do we square that circle?


Mrs Buick: That report was from 2009.


The Chairperson: Yes, it was from 2009; I appreciate that.


Mrs Buick: Those were given to you as case studies to show the improvement journey that those schools went on.  It is worth looking at.  The 2013 report showed that the school absolutely addressed those issues and had improved the quality of provision significantly by the time we had completed at least one follow-up inspection.  We identified issues around the quality of teaching and learning, but there were also issues around the direction given by leaders and management on teaching and learning.  It shows that the two are not mutually exclusive.  To have good teaching and learning, you need to have good leadership from the top, and if you read all the way through to the end, you will see that we ended up with much better education for those young people as a result of the catalyst of the inspection process.


Mr J Anderson: The principal changed there, too.


What you are reading in the evaluation of the quality of learning is inconsistency across a school.  In other words, there is some very effective teaching but it is not commonplace.  There is more ineffective teaching.  The question is this:  does the leadership know that?  It did not, actually, but,  if it did, how was it effectively addressing it to raise it through staff development, sharing of practice and showing and telling each other how to teach more effectively?  The leadership knew where the effective teaching was, but how was it reducing the inconsistency across the school to raise standards?  Another principal came in there and made a significant difference in a very short time because the person who was appointed had insight into the school and was able to have a running start, not a cold start.


The Chairperson: Did restructured management arrangements include self-evaluation?


Mr J Anderson: Yes.  In fact, one of the three areas for improvement identified in that report was to:


"develop a rigorous school development planning process"


We found that it was not rigorous.


"involving all the staff"


It was clearly not involving all the staff; it was written by somebody in a corner.


"which focuses strongly on effective learning and teaching"


Clearly, people did not know that there was inconsistency in learning and teaching.


"and incorporates a robust process of self-evaluation and review."


In other words, nobody was monitoring and evaluating effectively the inconsistencies of the learning experience as those pupils moved from classroom to classroom and got a good lesson one period and a poor lesson the next.  That was the story.


The Chairperson: The link surely was that there was an observation in the classroom.  That all came about as a result of that process.  If that is happening in only 12% of schools, why is this still an ongoing issue?


Mrs Buick: We see it —


The Chairperson: Not according to PISA in 2012.


Mrs Buick: That is just the schools that were part of that PISA survey.


The Chairperson: I accept that.  It was a sample.


Mrs Buick: We need to accept that that was a sample of specific schools.


The Chairperson: Have we any idea, Noelle, about the current percentage?  I assume — maybe I am wrong — that every inspection has an element of classroom observation.


Mrs Buick: Totally, yes.


The Chairperson: There is not one that does not have it.


Mrs Buick: No.  We always observe teaching and learning on inspection.  It is one of the key pieces of first-hand evidence.  I think that the PISA survey talked about observation in the school, not external observation.


The Chairperson: Surely observation within the school, as part of the self-evaluation, is desirable.


Mrs Buick: Absolutely.  We would expect that.


The Chairperson: But that is where there is a problem with the landscape.


Mrs Buick: We see it in more than 12% of the schools that have been part of the PISA survey.


Mr Moutray: This has probably been covered, but I still do not understand it.  At the end of the day, Together Towards Improvement indicates that evidence for the quality indicators should include classroom observation.  Whether the figure is12%, 15% or whatever, it still falls far short of what happens in England, where it is 60%.  From your response, I am still not clear what the issue is.


Mrs Buick: As far as I know, we do not have a quantifiable figure for how often it happens, but we would say that it is definitely a factor of good practice to have observation within the school to carry out self-evaluation.


Mr J Anderson: The answer to your question goes back to what Faustina said a short while ago.  It is about being clear about the purpose of that observation.  Observation occurs for every teacher twice a year through PRSD.  The question is about whether that observation is effectively helping to share practice in a way that will improve the consistency that we just described or whether it is done in a rather more mechanistic way that does not.


Beyond PRSD, schools that are effective have built a culture where there is an open door.  In those schools, there is an openness to teachers to see each others' practice.  There is an openness to show it as well as to tell it to teach other so that they can spread the expertise that exists within the school further and more consistently across the school.  That is the area in which further development is required.


In circumstances in which we have worked with schools where improvement has been effective, when the senior teachers have undertaken observation and we have undertaken observation and we have then shared our view of what "good" looks like, we have often found that there is a discrepancy in that the school regards a practice that it has seen as better than we might.  By giving an objective view of teaching, we help to lever upwards their expectation of what good teaching looks like.  We have seen that in some trails of schools that we have worked with over the last few years that have come out of a formal intervention and become "satisfactory" or even "good".


Mr Kinahan: I am very sorry that I was late.  One or two other things came up.  I have two questions.  One of the points brought up with the associate assessors who we had in was that, after an inspection, the most valuable thing is their work with other schools, with another teacher coming to talk to them and helping them learn the ways that they did things.  Are we going to put more resources into that so that it is not just self-evaluation at one school by the schools themselves but schools helping each other?


Mrs Buick: That is part of the work of the school development service, the setting up of which is in train.  As I am sure that you heard from district inspectors, they know where the good practice is because they are also the inspectors who carry out the inspection process.  They will often signpost principals and schools to where they have seen good practice.


Mr Kinahan: It just seemed an excellent way of doing it.  They were concerned that it was being cut back on because of resource issues and yet they found it to be the most valuable.


Mrs Buick: We have dissemination events.  We have just undertaken one for best practice in literacy and numeracy in post-primary schools, and we have another one coming up shortly.  Those events are about practitioners demonstrating their best practice to other practitioners, and they are very well received indeed.


Mr Kinahan: My other question follows up on what Stephen was saying.  How do you know that all this is going on in schools?  I did my little bit of maths at the beginning and found that, if you were to inspect every school, each school would be inspected every 16 years.  I got that from the number of school inspections being done in relation to the number of schools that exist.


How do you know that the self-evaluation and training is happening and is fully understood?  The TTI document is wonderful, but it is long.  Do you know that every school understands everything in it?  At the same time, how do you encourage everyone to take part in it?


Mrs Buick: Earlier, I quoted the statistic, based on NISRA feedback that is collected after an inspection, that 90% of schools found TTI to be very useful when carrying out their self-evaluation process.  The statistic that you quote is, I think, inaccurate.  Through our calculations, we determined that we will have inspected 90% of primary schools and 97% of post-primary schools within seven years.  So, we will have been round most of the schools within seven years.  Of course, we also have the district inspector activity, which is on top of all that.


The Chairperson: Noelle, there are many other things that we would like to stray into, but we will resist, because you will be back with us later in March.  In the meantime, thank you for the information that you have provided.  John, Heather and Faustina, thank you for your contributions.  Faustina got off reasonably lightly.


Mrs Graham: I am sure that you will make up for it.


The Chairperson: We will make up for it. 


We look forward to further engagement on the issue.  We will endeavour to bring our report to a conclusion reasonably quickly, because we feel that it is important to give some breathing space to everybody who thinks that, at the minute, we are prying into everything that is going on.  However, that is the duty and the role of the Committee.  We look forward to working with you in the future.


Mrs Buick: Thank you.

Find MLAs

Find your MLAs

Locate MLAs


News and Media Centre

Visit the News and Media Centre

Read press releases, watch live and archived video

Find out more

Follow the Assembly

Follow the Assembly on our social media channels

Keep up-to-date with the Assembly

Find out more

Useful Contacts

Contact us

Contacts for different parts of the Assembly

Contact Us