Official Report (Hansard)

PDF version of this report (223.28 kb)

Committee for Finance and Personnel

Review of the Efficiency Delivery Programme 2008-2011: Northern Ireland Audit Office Briefing

The Chairperson: I welcome Eddie Bradley, assistant auditor general, and Jacqueline O'Brien, audit manager, from the Audit Office.  Do you want to make an opening statement about the report?

Mr Eddie Bradley (Northern Ireland Audit Office): Certainly, Chair.  First of all, thank you for the invitation to present the findings of our report on the 2008-11 efficiency delivery programme.  If the Chair is content, I thought it would be useful by way of background to outline the context and why we decided to report on that area, and then to summarise the findings of the report.  At that stage, Jacqueline and I will be happy to take any questions and open up the discussion as members wish.

In 2010, the Committee for Finance and Personnel undertook an inquiry and published its preliminary report into public sector efficiencies.  The Audit Office provided evidence to the inquiry at that time.  In that evidence, we noted that the Westminster Public Accounts Committee had examined the accuracy and reliability of efficiency savings claimed by Departments in Great Britain.  We noted that among the main conclusions drawn by the Westminster Public Accounts Committee, there were doubts around the reliability of the claimed efficiencies; a lack of sufficiently robust data; a reliance on estimates; and inconsistency in the way that costs were treated.  It also expressed concerns that some efficiency projects may actually be having an adverse impact on service delivery and service quality.

There was a lack of scrutiny and challenge of the efficiencies reported.  There were particular difficulties in monitoring quality of service delivery and in ensuring that front line services were not being negatively impacted by the actions.  There were also failures by Departments to net off the upfront expenditure costs they had incurred in trying to achieve their efficiencies.  Consequently, they probably overstated the gains.

In response to our evidence, the Committee recommended in its report that the Audit Office review the performance of Northern Ireland Departments, and it was in that context that we undertook our study and that the Assembly's Public Accounts Committee referred our report to you.  It is fair to say that the main findings of our report broadly mirror what had been found by the Westminster Public Accounts Committee.

I will briefly summarise our conclusions.  There were some examples of good practice, where we felt that real efficiencies are likely to have been delivered.  They tended to be in areas such as improved procurement, efforts to improve energy efficiency and efforts to reorganise the workforce to improve productivity.  However, for around two thirds of the projects that we examined, we could not offer any assurance that genuine efficiencies had actually been achieved.  Around one third of the projects we reviewed did not have the basic financial and performance measurement information necessary for us to make an informed judgement on the achievement of efficiency savings.

The relationship between efficiency and quality of service is very complex.  Our view was that there was a strong risk that some of the projects that we examined either may not have been sustainable in the long term or may actually have had an adverse impact on quality of service.  We found that Departments, on the whole, measured only one side of the equation.  They measured inputs in cash terms, and there was little focus on the measurements of outputs, quality, or the extent to which front line services were protected.  This is key, because efficiencies cannot be validated unless Departments collate all the necessary information.

We found a number of examples in which Departments actually raised charges to increase revenue as a means of generating an efficiency.  In our view, these are not genuine efficiencies.  By and large, they are just passing the cost on to service users.

More generally, we found that the guidance developed by the Department of Finance and Personnel (DFP) at the time was not sufficiently detailed, nor was it fully implemented by Departments.  There was no centralised challenge function over the content of efficiency delivery programmes, nor of the validity of the efficiencies being reported.  Departments’ published efficiency delivery programmes were inconsistent; they lacked transparency about what was actually being proposed; they did not provide sufficient information to allow for meaningful public scrutiny or internal monitoring, and they did not provide an adequate basis overall for the measurement of efficiency savings.

The public reporting of efficiencies to the Assembly and more generally was not, in our view, sufficiently transparent, comprehensive or meaningful.

I hope that I have not spoken for too long, and that the overview has given members a sufficient feel for our findings and the key issues that we have identified.  Jacqueline and I are happy to take any questions from Committee members.

The Chairperson: Thank you, Eddie.  Have Departments understood what efficiency means?  It is clear that there does not seem to have been any progress made on this particular issue.  You also referred to the issue of charges being raised to meet efficiencies, which were then obviously not genuine efficiencies.  How much of the responsibility for that goes back to the Department?  On page 31 of the report, there is reference to the DFP guidance and three bullet points relating to the deficit in that particular guidance.  So, where does the blame really lie?

Mr E Bradley: I am going to sit on the fence a little bit.  I think it is a bit of both.  Certainly, the guidance at the time was not, in our view, sufficiently explicit.  There was a lack of clarity around definitions.  Where I have sympathy with DFP, however, is that it is not as though efficiencies, as a concept, were new.  There had been previous efficiency programme types, and guidance had been issued in the past, around 2004, by DFP, which was more substantive.

Nevertheless, I think it would have helped for the purposes of this programme had that been reiterated, so that the issue of just what was an efficiency was crystallised.  Notwithstanding that, I think that the Departments simply saw that their baselines were being reduced and had to live within those, and that was how they operated the thing, rather than taking a more proactive approach to generating efficiencies at a more strategic level.

We saw some good practice.  One of the health trusts, I think it was the Belfast Trust —

Ms Jacqueline O’Brien (Northern Ireland Audit Office): The MORE programme —

Mr E Bradley: — set up what was called the maximising outcomes, resources and efficiencies (MORE) Programme.  That was something where there seemed to be much better governance and much more leadership from the centre within that trust, and a much more genuine sense of a programme covering a range of different projects, aimed at delivering the totality of their efficiency savings.

The Chairperson: I turn to page 19 of the report.  There is reference at paragraph 2.9 to:

"A lack of basic financial and performance information".

A number of examples are set out there:  a lack of a baseline information; no link between the ratio of inputs to outputs; and insufficient information with regard to service delivery and performance.  That is basic stuff, is it not?

Mr E Bradley: It is fundamental.  Without getting too academic, you basically cannot have an efficiency unless you measure your spend, the outputs you are delivering, and the quality of service that is associated with those outputs.  Typically, what was reported was just a figure showing the amount by which the baseline budget had been reduced.  There was rarely any direct link made with the outputs associated with that budget, and in very few cases was there a link made to the quality of service delivery.  I suppose that the core definition of an efficiency is that it should not impact negatively on the quality of service delivery if you are reducing spend, nor should it impact negatively on the volume of outputs that you are delivering.  However, in very few cases did we see that explicit link being made.

The Chairperson: There is a very stark example on page 21.  This refers to the Northern Trust area and the closure of 28 beds at Whiteabbey and Holywell hospitals.  It closed 28 beds, but you do not know whether it has saved money in doing so.  It is shocking, to say the least, and that would certainly be the view of the public in those areas.

You looked at a number of different case studies overall.  As Departments have handled this so badly, in respect of defining efficiencies and acting accordingly, are there any examples where they have dealt with efficiency issues so badly that it has led to major inefficiencies and actual cost to the taxpayer?

Mr E Bradley: The difficulty we had was that because they are not measuring service delivery and volumes of output, we could not go that far.  We certainly felt that a number of individual efficiencies were just cuts to service.

There was also an issue, particularly in the health sector, in that we were concerned that what was purported to be an efficiency in one area would have knock-on effects.  I know that that happened in GB, for example, where they improved throughput by reducing the number of days that patients had to stay in hospital.  The corollary of that was that there was an increase in the number of readmissions.  So, by looking at a single efficiency measure in isolation, you are not taking account of the system-wide effects.  I think that there are big risks there.  That sort of partial review of an efficiency measure in isolation risks causing major strategic service delivery problems system-wide.

Mr Mitchel McLaughlin: This is a complex and difficult area, and we have visited it often enough to realise that.  The conundrum between cost-savings, efficiencies and a fall-off in services is the crux of it.  If the Executive, as a corporate body, are faced with a reduction in resources, then, per se, they are dealing with how to shave off the input in respect of resources.

I do not know what examples of different or better practice you have been able to look at.  I would have thought that the way to deal with the conundrum is to audit a particular service before you apply any cuts so that you then have a baseline.  That is your benchmark, and if you can do that for less, or do more with the same amount, that is an efficiency.  Have we tried that?

Mr E Bradley: In all honesty, we did not see that practice.

I agree completely.  To some extent, efficiency is a relative concept.  I think that there are a couple of ways in which you can try to effect efficiencies.  As you say, one way is to get the baseline in place and then, by measuring not just the cost side of the equation but the service delivery side, ensure that the latter is not affected by reducing the cost.

The other way is to try to benchmark with comparable organisations in England, Scotland, Wales and down South.  At least then you will have some measure of your relative efficiency vis-à-vis theirs.  Again, we did not see that done to any extent.

We have undertaken a number of studies in which we have adopted a benchmarking-type approach.  The first response is always that we are not comparing like with like.  I have sympathy with that, but I think that unless you are at least prepared to compare yourselves with others as a starting point, you cannot get a sense of where potential efficiencies are.

I think that monitoring, setting a baseline, and tracking change over time, together with benchmarking and comparing yourself with others, are the two fundamental tools that need to be applied in order to generate proper efficiencies.  In the current circumstances, in which public expenditure is reducing, you have to accept that some services may have to be reduced.  However, as long as it is done on a priority basis and is visible and transparent, then the rationale will be clear to elected Members and to the general public and there will then be a raison d'être for doing it.  We are not getting the sense in our review that those types of things were done.

Mr Mitchel McLaughlin: No, I could not see that in the report, and I assumed that it was because you were not able to identify certain things happening.

Is it possible to track the impact of cost efficiencies on the level of service?  In other words, could we identify whether services take longer to deliver or whether there is less quality in output?

Mr E Bradley: We effectively took a bottom-up approach and took individual efficiency measures.  A different conceptual way of doing it could be to take a top-down overview, almost a macro-level review, and use indicators from the Programme for Government and combine them with spend to try to get some sense in the round of whether public service delivery has improved despite facing the public expenditure cuts.  That is perhaps something that we will consider as an alternative type of output.

Mr Mitchel McLaughlin: I am not even going to go into performance bonuses.  [Laughter.]  You just wonder what they are delivering.

Mr McQuillan: The report is pretty damning, Eddie.  I feel that we need to hold people accountable for their actions.  How do you suggest we do that?  How can we put measures in place to ensure that people are accountable from now on?

Mr E Bradley: That is a difficult one.  This Committee, when it reported on the topic previously, recognised that the respective Assembly Committees could play a role and be a vehicle for leveraging some good practice into operation.

One way of doing that is to get efficiencies, or savings delivery plans, on to the agenda of Committees on a regular basis.  Having said that, we have had a look at the savings delivery plans, and a lot of the same issues are there.  Until there is a much more explicit link between expenditure and service delivery, the problem will not be cracked.  There needs to be much more focus on joining the two and not just looking at cutting a budget in isolation without clarity on what that means for the services provided.

Mr McQuillan: I was also thinking about arm's-length bodies, such as the Northern Trust, which we were talking about earlier.  That body is so far removed from us that we cannot scrutinise it.

Mr Mitchel McLaughlin: We cannot get at it.

Mr McQuillan: We cannae, and it is a body that I would like to get at.  How do you suggest we get a handle on what trusts and bodies such as that are doing?

Mr E Bradley: It is difficult for this Committee, which is at the centre, to get that level of detail while giving the individual Statutory Committees their place as regards their roles and responsibilities.  Can I be evasive on this?

Mr Cree: This is one of the most disappointing reports I have read for a long time, not from your point of view but in what it actually reveals.  In two thirds of the projects you examined, you can give no assurance of genuine efficiency.  That is a terrible indictment.

I take your point that the Department of Finance and Personnel may well have given insufficient clarification about what exactly was meant to be achieved.  It looks like the various Departments interpreted their instructions as being to reduce costs, and, therefore, missed the whole point of the exercise in my opinion.

Reduced costs can be achieved by simply reducing services, which is what you found in some cases, or by increasing revenues, which was a little bit cheeky as well.  For budget purposes, if you can understate your revenues and manage to realise more, you will have achieved a saving.  I am looking to see whether you have a clear definition of efficiency here, but surely the whole basis of this is to do the same job at a lower cost.  Is that a fair comment?

Mr E Bradley: Yes.

Mr Cree: So, Departments have singularly failed to do that.  Adrian asked what can be done now.  It is important that we realise that we have not got what we wanted, and the information that is there is just not good enough.  The trouble is that we are now halfway through the plan.  Is there anything else that can be done to rework the existing examples?

Mr E Bradley: There is an excessive focus on the expenditure side of the equation.  If there is one thing that the Finance and Personnel Committee could press on, it is to make sure that the performance element of the equation is brought to bear and is put on the table at exactly the same time, and that performance is not looked at one day and expenditure the next because they are intrinsically linked.  Unless the two sides of the equation are brought together, you will never get the full picture.

Mr Cree: That is probably a fair comment.

Mr Girvan: I apologise for not getting here a bit sooner.  As has been outlined, I think that this is a sad report, in that it shows that some people's interpretations of efficiency are totally different to what the reality should be.  I keep going back to what would happen in the private sector.  If you say to somebody there that you want efficiencies, they know that they will still have the same job to do but that they just need to learn how to do it more efficiently and either produce more for their time or cost less money.

Unfortunately, this report seems to be all smoke and mirrors.  Some people have taken their own interpretation of efficiency, and the baseline data is not accurate.  No one is recording it in the same way, and, because of that, in one organisation, and with the trusts, in particular, you do not actually know what you are seeing because they present it whatever way they want.

Whiteabbey Hospital is a perfect example.  What they deem to be an efficiency is to close a ward, but what does that do?  It does not mean that any more patients are being seen, and it does not save money.  How did they interpret that as a saving?  It is not an efficiency, and, to the general public, it is seen as a reduction of service, never mind anything else.  I think that some of those people need to be educated as to what is meant by efficiency.  It is not a matter of them coming and telling us that they have made savings when, at the end of the day, in reality, that is not what has happened.  There needs to be a bit of straight talking.

If some of the people who were in charge of that so-called efficiency delivery service were in the private sector, they would have been out the gate on Friday afternoon with their P45s because they would not have delivered.  It does not mean employing another 10 people to ensure that we are having efficiencies.  These are people who are brought in to monitor.  It should be very clearly demonstrated by what is coming forward.  So, it is a matter of going back and saying that they are not demonstrating the savings that they have been told to bring forward over the next number of years.  It is definitely not there.  I think that we have to go back to each Department and make them aware of exactly what is supposed to be delivered by this efficiency.  It is not a reduction in service; it is delivering the same, more efficiently, and for less money.  That is what we are trying to do.

Mr E Bradley: It is fair to say that, on the ground, we found that there was a lack of understanding conceptually of what an efficiency actually was.  If you step back from it, another difference that we found of how the efficiency programme was run in England was that there was a very strong central challenge focus to ensure that proposals that came up to the Treasury or to the Office of Government Commerce (OGC) were challenged to ensure that they were genuine efficiencies.  DFP did not see it as its role to exert that particular function.  If there is no challenge function in the operation and roll-out of the programme, it will lend itself to imprecision.

Mr Girvan: Why was it not seen as a priority for DFP to deal with this on that basis?  I ask this simply because I thought that it should have been a key function of DFP to ensure that each Department was doing that.  I appreciate that this is a fairly recent report that only came out in December and identified that.  However, now that we have got it, we will have to bring them all back, crack the whip, and by that I mean informing them that they are definitely not playing ball.  Some of them are trying to tell us that they have made efficiencies, but figure 4 on page 17 shows that as being unlikely.  However, they were claiming that some of the "unlikely" ones were showing efficiency.  Only a very small percentage actually had any efficiencies that were real and delivered.  We need to bring them all back, and DFP should be using its authority, as the body that gave out money in the first place, to ensure that what is being asked for is being delivered.  That is why we, as a Committee, should probably put a bit more focus on that.

Mr D Bradley: Good morning.  I listened to the Minister in the House recently answering questions about cutting services and increasing charges for some services provided by his Department.  His answer to questions was, "I have to make efficiency savings".  That attitude seems to match what you are saying in the report.  You also say that postponement of capital programmes should not be viewed as an efficiency saving either and say that there is a need for DFP to ensure a clear definition of efficiency savings.  Do you have a clear definition of it?

Mr E Bradley: As an Audit Office?

Mr D Bradley: Yes.

Mr E Bradley: Yes, I think that we do.  Up front, we set out a number of principles that are reasonably well established from a number of reviews undertaken by the National Audit Office, the OGC and the Treasury.  Even DFP, in the past, has signed up to most of them.

Page 10 has a series of bullet points, and those are pretty much accepted as being what defines an efficiency.  As an office, we are not allowed to question the merits of policy, but I can understand why charging fees or increasing charges might be seen as legitimate from a policy perspective.  It can lead to the more efficient delivery of a programme, but doing it on its own is not an efficiency if you are just transferring the cost to the end user.  DFP has signed up with us to say that those seven bullet points are widely recognised as the definitions of an efficiency.

Mr D Bradley: Does the sale of departmental assets come within the definition of efficiency?

Mr E Bradley: If there are underused or underutilised assets, it can be more efficient to sell them rather than hold on to them and probably pay maintenance, repair costs, and so on.

Mr D Bradley: Even though they are sold to other Departments in some cases?

Mr E Bradley: That is a moot point.  As far as postponing capital expenditure is concerned, to be clear, I think that there are times when assets can be used more productively and their lives extended slightly.  That is probably more efficient.  However, in one or two cases, we found that where you are simply cutting your capital budget and are not undertaking newbuilds, and where instead of doing something every five years, you are postponing it until 10 years, you are doing so merely to stay within budget as opposed to delivering genuine efficiencies.

Mr McQuillan: I have a quick question.  In answer to Paul, you said that there was a lack of understanding on the ground about what was supposed to happen.  The people who should be ensuring that this will happen are the people who will probably be getting a fair size of a bonus.  How can you equate that with this?

Mr E Bradley: One thing that we found was that Departments were not particularly leading their arm's-length bodies.  They were simply passing the reduced budget on to them and leaving it at that.  We found that there was a lack of leadership, a lack of robust guidance and a lack of challenge from the central Departments that had arm's-length bodies.  In one or two cases, we found that the central Department did not know what its arm's-length body was doing to deliver the efficiencies.  I do not wish to use a sledgehammer to crack a nut, but there is, clearly, a role for central Departments to get more of a handle on what is happening on the ground.

Mr McQuillan: That certainly comes across from the report.  Thank you, Chair, for letting me back in.

The Chairperson: Thank you very much, Eddie and Jacqueline.