5 March 2009

Do you believe the Audit Commission?

Lewisham was awarded 4 stars today by the Audit Commission in its Comprehensive Performance Assessment (CPA).

The Audit Commission says in its summary of Lewisham:

"Lewisham Council is improving well. Lewisham’s performance is improving in priority areas such as tackling crime and the environment with reductions in robberies and burglaries and improved street cleanliness. Improvement is maintained in services for adult social care and children and young people, especially for vulnerable young people. Overall the rate of improvement on performance indicators is below the national average and a below-average proportion of indicators is in the top quartile. Local survey results show improvements in satisfaction over the last year. The Council operates at the highest level of the equality standards and provides good value for money. The Council has good capacity to achieve continuous improvements in services through its clear planning processes and robust performance management systems. Partnership working continues to be strong and the jointly-run Downham centre has improved access to health, library and leisure facilities. The Council has robust plans to improve customer access to its services. However, challenges remain in further improving outcomes to above national averages and in meeting national targets such as the decent homes standard and recycling rates."

So lots of good stuff there, and credit to the staff who have brought about the improvements highlighted above.

However there is an increasing body of opinion that states that 'target-culture' actually corrodes public services and makes them more distant from the people they are supposed to serve. Simon Caulkin has written on this subject on a number of occasions in his management column in the Observer - here are a few:

Targets can seriously damage your health

Police bureaucracy that needs to be arrested

Of particular interest for Lewisham given the Mayor's previous ambitions to getting massive 'efficiency savings' by restricting access to adult social care is a general thought as to how bureaucracy and targets can hamper the effective delivery of the right social care to the right people:

We can still defuse the ticking care timebomb

And on a more recent and very worrying theme, we have Baby P

Blame bureaucrats and systems for Baby P's fate

In Camden its 4* rating was clearly viewed with some sceptism by the local paper, teh Camden New Journal

Top marks all round for the Town Hall but is it simply the best?

This article in the Camden New Journal starts :

"IT might not mean much to someone stranded on the 15,000-long waiting list for a council home. Not much either to parents struggling to find a secondary school for children in the south of the borough....But, stop rubbing your eyes, when it comes to the local authority league tables Camden, it was revealed on Tuesday, is ahead of all the rest. Top of the heap. A-number one." [Camden New Journal,08 May 2008]

And in some respects I am sure those concerns resonate with many in Lewisham.

I think the point being that from two angles there is evidence that the targets culture is harming delivery of services as viewed by Simon Caulkin, and further that it is harming the relationship between local government and the people we serve.


Andrew Brown said...

It seems to me there's always like to be a tension between the findings of an external audit and the lived experiences of some people who receive those services.

I'm less sure that targets or performance measures are corrosive of public services as you argue. For example, when I was a councillor, I always found it helpful to know how many times a child in care had been moved in the course of a year; because the life chances of those children were enhanced if we tried to reduce them. And I think that during my time on the council we did reduce the numbers that were moving too often. As this was a performance indicator that the government set you could see this as a target that distorted/changed the way that a public service was being delivered. I prefer to see it differently, and maybe those children who had a bit more stability in their lives saw it differently from me.

Of course the social workers had to make professional judgements on each case, and sometimes (too many times) placements broke down. For those children the fact we made our target for not moving children won't have mattered; their lives were thrown upside down by changes of foster carers, or residential homes, schools and friendships.

But overall, I can't think that having that target was wrong, or that the council was judged on it.

Cllr Dean Walton said...

One of the problems with targets is that they do a number of things.

Distract managers/professionals from delivering service improvement to instead meeting the targets. Now that doesn't sound too bad - but the example that we see frequently in the papers is 'teaching to test' that seems to occur in schools - a clear and direct result of setting such targets.

Another example woth looking at what might happen in a general housing benefits service - the target is to make decisions within 30 days of application. However, someone who can provide all the relevant information at the time of application should have a pretty much instant decision. Yet as the process is about reaching a decision within 30 days and not as quickly as possible we get a system whereby applicants deliver documents that are then processed by backrooms, inefficient transfers, follow ups etc occur and at some point a backlog arises resulting in drafting in loads of staff; the service allocates staff to dealing with the backlog, refuses to take telephone calls, more letters get written...does that sound daft or familar - yet the service is considered to be good. If instead of '30 days' we had a system which allocated caseworkers to cases and encouraged them to work with applicants then we could far more of these dealt with first time. An initial interview with the applicant with a view to actually deciding their claim at the end or very shortly afterwards.

Some people (quite wrongly of course) feel the need to inaccurately report performance against statistics - meaning that problems are hidden. The 'figures' are OK, but the experience of local people does not match those.

Setting the targets brings to the fore a whole industry of compliance which is wasteful.

Finally, I am not against targets per se, but these need to be meaningful and set by managers on the ground as part of their management of the service. Targets set by political leaders have a high chance of distorting the service away from what it should be and further giving confidence that it is good, when it isn't.

I know that politicians from all parties like targets (reduce CO2 by this, increase school grades by that etc etc) but unless they are backed up by some genuine evidence as to why it should be so much, it's arbitrary. So reduce CO2 by X% by Y date to avoid the worst excesses of man-made climate change is OK; increase schools grades by Z% to...there isn't an answer to that type of question. It's far more honest to answer that and state clearly "We will build new schools as money becomes available and provide headteachers with the money they need to run good schools. Governing bodies are encouraged to work with headteachers, students and parents to develop range of ways of monitoring success in the school." This then means that the professional judgement of educational acheivement is allied with the aspirations of the governing body. If XYZ school beleive it is important for all children to get so many GCSEs at such and such grade - great - they can have the 'teach to test' style - if not, then they don't need it.

Andrew Brown said...

I don't think there's any doubt that targets can create perverse incentives, but not having them can also mean important stuff doesn't get done.

So, to continue to use my example of the placements for children in care, while I'll concede that by having the target some children were no doubt left in placements that were unsuitable. But before the target was introduced by Ministers lots of kids in care were being moved way too much.

I'd also make the case for political engagement in target setting. It's part of the responsibility of being part of any executive that you set the targets that your organisation works to, politicians shouldn't be exempted from that.

I also think it's perfectly acceptable to set outcome focused targets, as long as the resources are there to meet them and they're flexible enough to allow for local innovation.

Of course politicians shouldn't set targets without drawing on advice of those who'll be delivering them. But I think there has to be clear accountability for outcomes as well as inputs.

Cllr Dean Walton said...

If targets are the right way to manage, control, influence an organisation I would agree.

The problem with targets is that interfere with the ability of services to deliver. They are useful in explaining to the public at large what's happening/what they might expect and for using on a political platform. Yet what's important to people is whether they get access to the services they need, the services they want at the time they want them. If, at the point that someone's parents needs care, it is acceptable for them to wait for 5 days for an assessment meaning that person has to take time off work, etc etc then that's not really good enough. Targets such as this are in fact arbitrary. Looking at Lewisham for instance, managers should generally know how many referrals/contacts etc we should get in a year. The challenge then for managers is how to meet that demand within available resources. Exalaining to the politicians what the likely impact of a particular level of resourcing is. The arbitray targets are a 'get out' - they allow services to be described as good when that might not match the exerience of local people.

Andrew you will know that one of the key services for Lewisham is the bins...how many bins can be missed before the system gets described as being poor? Is it acceptable that 2% of the bins are missed every week, or is 3%...or should it be that "we aim to empty every bin every week and if we miss you one day, we will try to collect you that day or if not the next day"...there are crews of Lewisham staff moving around Lewisham all the time there is no reason why a single missed bin could not be collected within hours of it being reported - delighting our residents and not adding much cost.

Another good example is one that you're familiar with - Love Lewisham. It's great that people can report things and they get sorted out...people don't expect graffiti to go immediately but if they report it and within a day or so it's done - and the managers are able to deal with the issues in an efficient way. If a target was set you might find on a particular day a graffiti team is called up to deal with stuff in Downham and then stuff in Evelyn using up a massive chunk of time travelling between them.

I'll leave it at that for now - this discussion can rage on. Feel free to respond to this, but I won't be replying. We need to discuss this type of thing face to face.

Needless to say there are some interesting books on this topic.

Andrew Brown said...

Don't worry I'm not going to push it.

Freethinker said...

Interesting discussion.

I would have a look at The Systems Thinking Review.

It has articles and videos that highlight the gap between what the performance measures say and the experience of service users.

In short ...the audit commission are clueless and inspection damages learning and improvement